Just sharing a new engineering blogpost on How we built a queue on top of Kafka
Quickly, a little bit about me. I’m Constantijn. I joined Monzo in 2017 and have been part of our Backend engineering discipline ever since. I’ve worked on many of your (hopefully) favorite features in that time, but that is for a different discussion.
Back to the blog post - Monzo introduced Kafka to its technology stack around the time that I joined the company. By pure coincidence I was on the first product squad (one of only two!) to use Kafka for real. I felt first hand the pain of trying to use the library we’d crafted to use Kafka as a queue, though we didn’t consciously realise that’s what we were doing at the time. First I made a few small changes for my own benefit and before I knew it I was a primary maintainer together with Kieran Gorman who had done the same thing. Fast-forward to now and in many ways the two of us still are. This blog post is something I’ve wanted to share for years and covers core parts of the journey we embarked on when we both naively raised our first “small change” pull requests.
I’m excited to hear what you think and answer any questions you might have.