The architecture and operation of a data pipeline in cloud is a very different exercise compared to setting up traditional on-prem ETL processes of the past.
Let’s dive deeper into the architectural patterns (and antipatterns) of cloud when it comes to setting up data processes. We’ll look at the technical considerations and some caveats you might encounter when building in cloud.
This will be a technically oriented session, basic knowledge of cloud products (VMs, Redshift, S3…) will help you fully benefit from the session.
Join and learn about:
- What it takes to set up a production data pipeline starting from zero – the cloud components to use and why (using an example in AWS)
- We’ll show and explain an example architecture of a data pipeline in the cloud
- Estimating costs and how to avoid overruns
Kevin Scott's long running tenure with CloverDX includes numerous complex data consultancy projects and working with fresh customers and helping them succeed on their journey.
CloverDX has been working at the coalface of data challenges since 2007. We’ve worked with companies large and small around the world, in industries such as finance, healthcare and consulting, to help them build and manage better data pipelines and deliver better data outcomes.