From its first release in 2019 to managing over 20,000 clusters, IONOS scaled its Kubernetes service by building on a massive etcd foundation.
The team progressed from the CoreOS operator and Bitnami Helm charts to a simplified custom Helm chart for better control and stability.
Using a shared etcd with client-side prefixes reduces cost but creates noisy neighbor problems, requiring careful tuning like compaction and defragmentation.
Initial cross-location clusters suffered from latency and revision drift, leading to a more stable single data center layout using availability zones.
A live migration process using `etcdctl mirror` allows moving a Kubernetes control plane to a new etcd cluster without global downtime or data loss.
By modifying an etcd snapshot to insert a high revision number, clients like kubelet continue watching for changes without needing a restart after migration.
The team is working on automating the migration process, offering dedicated etcd clusters, and contributing their migration learnings to the Kaji project.
27:52Maish Saidel-Keesing
29:21Um e Habiba
57:24Hannes Norbert Göring
42:45Marc Nimmerrichter
19:36Igor Khokhriakov
30:35Michael Cade
28:36Noaa Barki
48:45Denis Souza Rosa
.gif?w=240&auto=compress,format)
.gif?w=240&auto=compress,format)
Jobs that call for the skills explored in this talk.
IONOS SE
Hannover, Germany
IONOS SE
Brunswick, Germany
IONOS SE
Bonn, Germany
IONOS SE
Mannheim, Germany
IONOS SE
Mainz, Germany
IONOS SE
Aachen, Germany
IONOS SE
Dortmund, Germany