What's in this webinar?
Chapter 1: What is data ingestion? (0:10)
By data ingestion we mean pulling data from source and pushing it to your target database with little to no remapping.
/2021-10-26_01%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_01%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Chapter 2: Data location (1:49)
What locations are you trying to combine? Cloud or on prem? In files, databases or in applications? All that requires thinking about security, access permissions and even combining multiple ecosystems together (e.g. AWS to Azure).
/2021-10-26_02%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_02%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Chapter 3: How to map your data? (5:25)
What tools can you use to get your data to where you need it? Which are best for you - scripting, dedicated data loaders or a full data integration solution?
/2021-10-26_03%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_03%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Chapter 4: Overview of the CloverDX Data Management Platform (7:43)
An example of a full data integration solution, CloverDX allows you to connect and automate ingestion of data from anywhere to any target, combining cloud and on-prem.
/2021-10-26_04%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_04%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Chapter 5: Data push vs. data pull, which one? (10:10)
With Data Push, processing happens close to the source and you don't need to open your data source. Conversely, Data Pull works well with large volumes where ELT workloads make more sense.
/2021-10-26_05%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_05%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Chapter 6: Moving your files to cloud (13:50)
Why keep files in the cloud? Blob, block storage vs other cloud file storage options
/moving%20files%20to%20cloud.png?width=500&height=281&name=moving%20files%20to%20cloud.png)
Chapter 7: What moving files to cloud looks like in CloverDX (20:45)
A simple example of how CloverDX helps simplify and automate the file movement process.
/2021-10-26_07%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_07%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Chapter 8: Loading data to cloud databases (22:13)
Cloud databases come in many different shapes, sizes, and price ranges. Should you choose fully managed or self managed, do you need off the shelf or custom configuration?
/2021-10-26_08%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_08%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Chapter 9: Cloud warehouses, Redshift, Snowflake (28:08)
While they seem to behave like a regular relational databases, cloud DWH have quite different means of loading data into them - typically bulk loading through an object blob storage
/2021-10-26_09%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_09%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Chapter 10: Cloud data warehouses and CloverDX (31:12)
CloverDX provides simple native components to simplify to process of loading data into major cloud data warehouses, like Redshift and Snowflake.
/2021-10-26_10%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_10%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Chapter 11: Loading data into cloud apps (32:30)
Most cloud applications provide various levels of API access you can use. While appealing at first glance, APIs quite often come with technical caveats that need to be addressed, like rate limiting or high latency and need for batch operations.
/2021-10-26_11%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg?width=500&height=281&name=2021-10-26_11%20Data%20ingestion%20into%20cloud%20(821%205079%204182).jpg)
Build vs buy: Cracking the dilemma
Should you continue investing into your homegrown DIY data integration solution? Or should you bite the bullet and invest in a commercial data platform?
If your ETL processes aren't as reliable, transparent or easy to maintain as you'd like, it might be time to start looking for a more robust solution.
This guide will walk you through every aspect of the decision so you can make an informed choice.
