Ask HN: Have You Left Kubernetes?
Yes! My startup of 5 people did. We started out with a managed Kubernetes cluster on DigitalOcean, but there were a number of reasons that caused us to not be very comfortable with that setup.<p><pre><code>   - Taking random .yml configs from The InternetTM to install an Nginx Ingress with automatic LetsEncrypt certs felt not-exactly-great. It&#x27;s no better than piping curl to bash, except the potential impact is not that your computer is dead, but the entirety of prod goes down.
   - Because of this, upgrades of Kubernetes are a pain. The DigitalOcean admin panel will complain about problems in &#x27;our&#x27; configs, that aren&#x27;t actually OUR configs. We don&#x27;t know how to fix that, or if ignoring the warnings and upgrading will break our production apps.
   - Upgrades of Kubernetes itself aren&#x27;t actually zero downtime, and we couldn&#x27;t figure out how to do that (even after investing a significant amount of research time).  
   - We were using only a tiny subset of the functionality in Kubernetes. Specifically we wanted high-availability application servers (2+ pods in parallel) with zero-downtime deployments, connecting to a DO managed PostgreSQL instance, with a webserver that does SSL-termination in front of it.  
   - Setting up deployments from a GitLab CI&#x2F;CD pipeline was pretty hard, and it turned out the functionality for managing a Kubernetes cluster from GitLab was not really done with our use case in mind (I think?).  
   - It would be bad enough if DigitalOcean shit the bed, but the biggest problem was that we couldn&#x27;t reliably recognize if something was a problem caused by us, or by DO. Try explaining that one to your customers.
</code></pre>
Summarizing: it was just too complex and fragile, even once you wrap your head around what the hell a Pod, a Deployment, an Ingress and Ingress Controller, and all of the other Kubernetes lingo actually means. I suspect you need a dedicated infra person who knows their stuff to make this work, so it could very well make sense for larger companies, but for our situation it was overkill.<p>We were not intellectually in control of this setup, and I do not feel comfortable running production workloads (systems used by 20k high-school students, mission-critical applications used by logistical companies) on something we couldn&#x27;t quite grasp.<p>We went to a much simpler setup on Fly.io, and have been happy since. It&#x27;s a shame they seem to be too young of a company to really be super reliable, but I suspect this is only a matter of time. In terms of feature set, it&#x27;s all we need.