Looking back at your time with Monzo, what would you crown as your pride and joy in regards to a project you’ve worked on?
When I joined the Data Science team at Monzo, there were only 5 of us- so there wasn’t a machine learning team and everyone had to do a bit of everything. I’m really happy that a bunch of things I built back then are still useful to the new teams who have picked up those systems. The biggest technical project I was hands-on with recently was building a feature store. This is a system that now powers several of our machine learning models - so it’s great to see others build systems on top of the ones I’ve built!
How much has the full remote working impacted your workload during lockdown?
I’ve been grateful to not have to commute, but I think overall I’ve been working more - at least, it’s a bit harder to get away from the laptop in the evenings.
What’s your main downtime activity?
I am a purple belt in Brazilian Jiu Jitsu and absolutely love it although it was near-impossible to train last year.
What is the biggest limit on what you can do with Machine Learning right now? Legislation? Time/effort? SQEP? Computing resource?
Our biggest limitation is people - we have more ideas that we need to say no to right now than ideas we can say yes to. That’s why we’re doing a lot of hiring!
I didn’t watch the film but Hitchhiker’s Guide to the Galaxy remains the only book that I’ve burst out laughing while reading (I was on the tube and got weird looks)
What are your/Monzo’s goals when it comes to rolling out machine learning + AI in those areas, and what will that mean for the consumer, in terms of ‘adding more value’.
This is the type of question that I spend most of my time thinking about. Here’s one example from work last year: by using a combination of rules and machine learning, we built a system that could automatically route a chat to the right kind of specialist. How does that add more value? Customers, on average, would need to chat with fewer agents and so they would get to an answer faster. For Monzo, it’s more scalable to have one person work on a chat rather than two (so we can help more customers). So it’s a win-win all around!
What approach do you take to finding the right balance of machine learning algorithms and the human element? Do you see machine learning as a means for replacing some human elements, or as something that can augment them?
For areas like customer service, machine learning does not replace a human touch – it helps to get you there faster. For other areas, like detecting and preventing financial crime, automated systems will work non-stop to protect you, and so will do a better job than if we had humans!
What are your thoughts on how other banks approach Machine learning? What have others in the industry done that you think monzo has done, or could do, better thanks to machine learning?
I haven’t been very good at keeping abreast of what other banks are doing with machine learning- sorry! One bit that I do know is different is that our Machine Learning Scientists are empowered to work on machine learning systems end-to-end. In other companies, there is often one person/team who trains the model and another person/team that ships the model into production. We don’t do that: we upskill our ML Scientists to work on our production code safely. From this we’ve seen a huge speed up in the time it takes to test an idea.
Is an AI chatbot something Monzo have explored, or planning to do, to serve as as a concierge between FAQs and human based support? How easy do you think these systems should be for users to bypass the AI and speak to a human? A shortcut (/human) like N26, or click buttons and hope for the best like RBS?
We experimented with an automated responses for some queries in 2019 – back then, the bot was called Monzo Helper. While it showed incredible promise, there was a decision to not pursue this direction any further in 2020. One of the motivating reasons for that is that there are many problems that customers should be able to resolve by simply tapping a button, rather than needing to chat to a bot. And so the teams refocused on improving our app on that front.
Yes! It’s made me a lot more aware about not making assumptions about people’s names - I know how it feels when I need to ask people (sometimes, several times) to spell my name correctly
We have our best and brightest working on this space!
We don’t use any machine learning in this space at all - this is a great example of an area where defaulting to an expert human’s review is the better thing to do.
What’s the biggest AI/machine learning mistake that’s been made in your time at Monzo (or career more generally?)
(And by that I mean, when an AI has just made a mess in a “oh we didn’t expect it to work like this way”, rather than a “it was a mistake to do AI on this task” kind of way, if that makes sense?)
A few come to mind! The first is not really an AI fail, but I once broke our help article system and it would just… quietly not return any results. One of our QA testers picked this up quite quickly and my heart sank as I quickly rolled back the change I had made
Maybe more of what you are looking for: about a year or so ago, we ran an experiment on an old system where we used machine learning to try and find a better order for the chats that are our agents work on. The idea was that answering similar questions in quick succession (e.g., a bunch of customers who all want a bank statement) would be faster than getting agents to jump from one type of customer query to another and needing to context switch every single time.
The experiment showed some very promising results, but one thing that we had forgotten to factor in is that our expert agent look for patterns in the customer queries they are getting and report unusual trends to their managers - after all, it may mean that something has inadvertently broken! So we had panicked COps reporting in that there were unusually large volumes of requests for bank statements, and others reporting in unusually large volumes of lost cards – when in fact all that was happening was that our system was splitting these chats up and sending them to agents who had recently worked on similar questions. Oops!
We moved quickly to fix this - but in the end, we moved a different direction with this overall system.
Tonight and tomorrow will be your last chance to get questions in for @Neal who has been amazing and been jumping in all week to pick them up. Absolute hero!
Ok so I’m getting a little meta now, but I put the entire content of this thread into a GPT-2 algorithm online, and started typing questions to see what the algorithm thinks it should ask…
What is your favorite feature for the app ?
If you could go back to any point in time and change one thing about Monzo , what would it be?
If Monzo was not around today , what would be your ideal future vision for the company?
Would you rather have a free version of the app , or an ad -supported version that includes more features for no additional cost?
The green bits are what the algorithm said… [I was trying at the end there to see if the algorithm would ask about fingers for toes/etc]
Ha! This is the first time I’m answering to a bot!
In terms of impact that it has had on me, personally: the combination of paid early + salary sorter. This was an absolute game changer for me and I basically stopped using other bank accounts after this.
In terms of impact more widely: the gambling block. Hearing about how this helped people to get out of very tough times was absolutely heart wrenching.
That would be the best question I have ever had to answer. But I think it’s easy to imagine Monzo moving forward as one of the pioneers in the world of automated brokerage and trading services. It’s definitely not a company that’s just going to sit back and watch as other people do it better and faster.
If Monzo was not around today, there would be nothing to do except envisioning it existing?
Hrmm… I don’t think anyone would like ads in the Monzo app. I wouldn’t. I’m all for morer insights, offers, premium features, and recommendations - anything that gives customers value while making Monzo sustainable - but not ads.
although it was near-impossible to train last year.
accounts? 
[I was trying at the end there to see if the algorithm would ask about fingers for toes/etc]
If Monzo was not around today, there would be nothing to do except envisioning it existing?