In February 2017, Jacqueline Xu of Stripe published Online migrations at scale. It describes moving hundreds of millions of subscription records from one data model to another while the Subscriptions API kept serving traffic, with no maintenance window and no flag day. The four-step pattern it lays out has become the standard answer to "how do you change a live data model?", and it applies whether the data lives in Postgres, DynamoDB or anything else.
What they had
Stripe's subscription data had already evolved twice:
- At first, each customer could have at most one subscription, stored as a single field on the customer record.
- To support several subscriptions per customer, the field became an array of subscriptions, still stored inside the customer record.
That second design worked, but it tied two things together that change at different rates. Any change to any subscription meant rewriting the customer record, and any query about subscriptions meant scanning customer records. The goal was to move active subscriptions into a table of their own.