Hey everyone, I’m Will I work in the Backend Platform team at Monzo.
My colleague Joseph and I have written about how our team brought automated rollbacks to our deployment system. This is the most substantial change we’ve made to our deployment system in some time, so it was not without its challenges!
At the heart of this new feature is Argo Rollouts - a Kubernetes extension that supports advanced deployment strategies. In this post we dig into how we integrated Argo Rollouts with our existing deployment tooling, while keeping the Monzo delight factor. We show how we migrated all 2,000+ services to this new system and discuss the lessons we learnt along the way.
I wrote a bit about engineering effectiveness metrics we track here - there’s definitely room for improvement here. For those kind of metrics we use the data in our data warehouse (BigQuery / dbt / Looker).