Very Robust etcd


(Naji Esiri) #1

#2

I didn’t get all of that, but it’s nice to see monzo have proper well thought through back up plans.


#3

a great write up by Filipe even if it goes over my head at points


(Jake Tame) #5

Awesome post and wish I could do this kind of stuff!


(Rory McCune) #6

interesting stuff. One thing, from https://github.com/monzo/etcd3-terraform/blob/master/cloudinit/etcd_member_unit it kind of looks like you’re using etcd without SSL?


(Oliver Beattie) #7

Good point. As the README of the repository makes clear, this is a deliberately simplified deployment designed as a demonstration:

:warning:️ Warning
This is an illustration of how we run etcd clusters. This should not be deployed to production without first securing the infrastructure.

Since TLS wasn’t necessary to demonstrate these features, and would complicate people’s experience deploying a demonstration stack, we didn’t bother with TLS in that repository. You should not run this in production as-is!


(Rory McCune) #8

Cool. Probably worth having for etcd, even in an MVP as without it there’s no authentication and anyone who can hit the port can dump the entire cluster config (secrets and all) in one command…


(Jose) #9

hey, great article, thanks! Just out of curiosity, did you consider to use something like etcd-operator, as they already solved some of your requirements?


(Filipe Varela) #10

:wave: The etcd-operator does something very similar but is designed to operate etcd clusters running atop Kubernetes. This etcd cluster is the one that actually backs Kubernetes and although it is possible to self-host, this is not something we do at the moment. We do plan to use the operator for etcd clusters which back some of our services locking operations though!


(Jose) #11

hey @filipe, yes, It makes sense as self-hosted solutions are not GA yet. Thanks for the answer! :slight_smile:


(kalyan) #12

The etcd-operator is meant for operate etcd cluters running atop kubernetes. etcd is a consistent and highly-available key value store used as Kubernetes’ backing store for all cluster data.

Always have a backup plan for etcd’s data for your Kubernetes cluster. For in-depth information on etcd. Operating etcd with limited resources is suitable only for testing purposes. For deploying in production, advanced hardware configuration is required. Before deploying etcd in production