How I Proved a 12-Million-Record Data Migration Was Actually Correct
The first time my team compared our migrated data against the original, only 86% of records matched on the field that mattered most. By the final run, 99.9994% of 12.3 million records matched, and we could explain every record that did not.
Nothing in the gap between those two numbers involved moving data. All of it was verification work. That is the part of a migration almost nobody plans enough time for, and it is the part I want to walk you through.
I have spent two decades building and leading engineering teams, most of that time in digital marketplaces for health insurance. In that world, a record is rarely just a record. It can be a person's decision about whether they want to be contacted. When we moved data off a legacy Oracle monolith into microservices backed by PostgreSQL and DocumentDB, "the rows arrived" was never going to be...
Copyright of this story solely belongs to hackernoon.com. To see the full text click HERE
