RESOLVED: Monzo Services Degraded/Outage - Card Payments may fail (16/01/18)

And this is why I believe in Monzo.
Humans screw up sometimes, that is inevitable, I have huge respect :fist: for a company that can put they’re hand up and admit to making a mistake, explain clearly (even if it goes over my head a bit) why it happened and what they’re doing so it doesn’t happen again.

Now if we could only get our politicians to admit when they were wrong, we’d be a lot better off :yum:

4 Likes

Some other banks should take note on how to inform their customers instead of hiding behind the curtains :eyes:

At least it’s fixed and lessons learned though :slight_smile:

Yes I am talking about them :joy_cat: :tea:

7 Likes
  • Are you going to monitor pausing?
  • Do you have anything neat like LastFM?
2 Likes

Are you going to monitor pausing?

Yes, we have shiny new graphs and alerts just for this.

Do you have anything neat like LastFM?

We do have dashboard TV’s in the office, but not related to NSQ at the moment. This is a good idea though!

Awesome! Others should look to you as a model. When one of your competitors had an issue recently, they pretended it never happened and their CEO resorted to personal attacks. Go Monzo and team!

4 Likes

So paraphrasing (and oversimplifying) to make it clearer for a non-technical audience

On the 15th you experienced overloading of NSQ so you upgraded part of it. On the 16th you proceeded to upgraded the rest of NSQ, made a mistake and as a result overloaded a number of services

You’ve now have monitoring in place to make it easier to detect NSQ overloading and processes in place to make it harder to make the same mistake again

Is that correct or have I misunderstood?

2 Likes

Eliding some context I think this is accurate. :+1:

3 Likes

Automate all the things! :metal: Don’t forget scalability, reliability, availability, security and everything nice… :sleepy:

1 Like

If you search Twitter for mentions of said CEO, there appear to be clusters of coordinated praise with similar timestamps and wording from different accounts of real people. Good PR dept / CPO there.

1 Like