What is data migration? Making original systems data fit a purpose in a new system. It might take weeks to months or even years to complete,
Strategy, Planning, Implementation, Execution - how to phase your migration project and why this organization matters.
Answering the fundamental questions that shape your project - data, environments, dependencies - inform how you'll carry out the migration - either big bang or trickle style.
A very important aspect of planning - identify and AGREE what's part of the migration to avoid future surprises. Scope also determines the duration of the project and its requirements on management strategy.
While not necessarily needed as a Gantt-style chart, think of both happy and unhappy paths in order to anticipate problems and plan for contingencies.
Another step is getting deep into the technical aspects of the project - what connectivity you'll need depends on the variety of systems you'll need to pull data from and push to.
Will you need to take precautions when handling some sensitive data (for example GDPR, will you be sharing the data with a 3rd party or overseas, be it for a brief period of time)?
Finally data quality plays an important role, data migration often being the long-awaited trigger for a data cleansing project.
Should you choose to develop the tooling for your migration yourself or use an off-the-shelf product?
The trade-off often comes between the price of the additional tools vs the experience and cost of the developers that is considerably higher with home-grown scripts and tooling than with commercial products.
In our experience, better to plan having a smaller, better skilled and aligned team than trying to solve a big problem with a big team.
Data mapping is a key activity and artifact in a migration. It needs to be properly managed as it evolves throughout the project cycle, as well as needing to be understood by everyone involved.
Factor in known unknowns. Things will change, stakeholders will change their mind.
Despite the temptation to start easy, we recommend you look at your most problematic and most complex data first. This approach will allow you to quickly uncover potential problems with scope, resource and deadlines, allowing you to reassess the plan early.
Test everything you can, as often and as early as possible. Don't underestimate testing and don't leave it to the very end.
Now that everything is ready, it's time to pull the trigger. Communication is key here - reports on what's been migrated, what's working and what's not. You should be ready to provide some temporary hypercare so that everyone involved has somewhere to report problems or ask questions.
Manual processes always end being very costly - always implement your migration with as much automation in mind as possible. It will pay off in the end, especially in larger trickle migrations.
Sometimes neglected but very important in communication with stakeholders - be ready to show exactly how you measure success and where you are at any given moment.
How can the CloverDX Data Management Platform help you with implementation and automation of a data migration?
We've helped many organizations migrate their data and systems, small and large. Heres a few case studies you can look at.
Should you continue investing into your homegrown DIY data integration solution? Or should you bite the bullet and invest in a commercial data platform?
If your ETL processes aren't as reliable, transparent or easy to maintain as you'd like, it might be time to start looking for a more robust solution.
This guide will walk you through every aspect of the decision so you can make an informed choice.