How to Get Data To/From AWS Databases

Let’s begin with a hard truth. Many cloud providers, including AWS, make it their aim to get your data into their databases and keep it there.

As far as Amazon are concerned, they’re the only cloud database provider on the market. From their perspective, it's like Hotel California: you can check out any time you like but you can never leave. Of course, this is far from the truth. So, let's look at how to get data to/from AWS databases.

Automated data migration and AWS database

In fact, migrating some or all of your data to an AWS database is actually quite simple – and there are plenty of databases to choose from.

Amazon offer transactional databases that capture and store data, as well as a range of data warehouses that can analyse and spot trends in large quantities of data. These databases make up AWS’s ‘collect-store-analyze’ ecosystem.

diagram of the AWS 'collect, store, analyze' databases ecosystem

But, although it’s easy enough to migrate your workloads to an AWS database, it’s a painful process attempting to get your data back or integrate it with your multi-cloud environment.

In the following blog, we’ll be laying out how you can better manage your AWS database import, store and retrieval journey.

How to get your data to or from AWS databases

Get your data into an AWS database
Options for storing your data on AWS
Get your data out of an AWS database

1. Get your data to an AWS database

When migrating an existing small-medium database to or from an AWS database, use AWS Database Migration Service (DMS). This service allows you to move your existing data, full load, from its source point to the target database instance in a one-time migration.

Alternatively, you can import and export larger quantities of data using AWS Import/Export Disk or AWS Snowball.

Things become trickier once you need to upload daily batches of data regularly. Although AWS’s tooling is suitable for managing AWS data tasks, it doesn’t account for more universal jobs (i.e. uploading data from Azure or other competitor databases). As a result, your organization will need to orchestrate a process for daily uploads that isn’t AWS specific. This will usually involve using third-party ‘connectors’.

We faced these issues ourselves at Clover when we wanted to integrate some HubSpot data with some financial information to create some custom reports and we wanted to do it using AWS. See:Our Data Migration Process: How to Rebuild a Data Project in AWS.

2. Store your data using AWS databases

So, you’re planning a data migration to an AWS database. But which one do you choose?

We’ve listed some quick pros and cons of the main players:

AWS Database	Pros	Cons
S3	Developer friendly Easy to integrate within an AWS ecosystem Capable of storing large quantities of data Decently priced	Downloading data is expensive For further support, you’ll need to purchase the ‘AWS support plan’ Not easily integrated in a multi-cloud environment
Glacier	Built for housing large data backups cheaply ($0.01 per GB per month) It’s secure and simple	It’s expensive to retrieve your data
Aurora	It harnesses Amazon’s RDS database, a service that simplifies and automates time-consuming database management tasks Provides continuous scalability	For data that requires write updates against tables, Aurora’s performance may not be to standard
RedShift (See our article: How to load data into Amazon Redshift)	Fast performance A similar interface to PostgreSQL A range of integrated security features	Parallel processing/uploading is only supported by Amazon databases If you’re writing data from distributed, non-AWS systems, you have to handle the process yourself
RDS	Automatic backups, security and management High performance and capacity	Only compatible with Oracle or AWS relational databases
DynamoDB	Limitless scalability Integrated monitoring and management Predictable performance	Deployable only on AWS Cannot manage complex queries Expensive and low latency

3. Export your data from AWS databases

Now, here’s the tricky part. Once your data’s in, you’ll have to pry it from AWS’s hands to get it out.

Understandably, AWS don’t want you to migrate your data outside of their databases, nor do they make it easy to move your data between different databases. As such, extracting your data can be a difficult task.

For one-time large migrations, it’s likely you’ll need to move your data out slowly, piece-by-piece as to not incur large costs. Alternatively, for daily data chores and integrating your data between databases, you’ll need to find your own unique solution. This is a complex task that your data team might not be fit to perform.

Similarly to moving batches of data into AWS, you can simplify the process by using connectors that orchestrate your multi-cloud environment (and the data that moves in between your databases).

Info Sheet: CloverDX on AWS

Align your multi-cloud universe

Although getting your data to your AWS databases of choice is relatively simple, ongoing uploads, exports and orchestrations between different databases is difficult.

In order to negotiate your databases smoothly, you’ll need to find the right specialist data experts or invest in connectors that can piece together your multi-cloud ecosystem.

At CloverDX, we’re neutral in the ‘Azure vs. AWS’ argument. We are more interested in how you can make your cloud databases work together in harmony, wherever they are.

See how CloverDX can help your data migration

And if you want to know more, watch our webinar: Strategy for Data Migration to