Ask HN: Are we overcomplicating software development?
1) Yes, except that you should try some languages sometimes. E.g. if you use Spark in production as a critical part of your system... take the time to learn scala.<p>2) Is a pet peeve of mine. Theoretically microservices are good, but we don't have a way to orchestrate them. What's lacking (in programming languages terms) is a "runtime" and "debugger" and of course a widely-tested & reliable set of "libraries" for most common tasks. I think it's possible to do something like that, as long as you start imposing some restrictions on what a "microservice" is and how it talks to the outside world. Also, in this frame of thinking, it becomes apparent that "deployment" is actually "programming the system, at a high level". It's not "just configuration", configuration is code - if you moved the complexity from your "code" to your configuration, you just moved the complexity into a language that has very poor tools to work with.<p>3) My rule of thumb for most systems is "avoid redundancy in the control plane; you can and should have redundancy in the data/data processing plane; don't plan for 100% uptime in the control plane, plan for very short downtimes/ for fast recovery when something goes wrong"<p>4) My experience is that continuous integration is good for all but the very small teams.  For multiple reasons, not just "microservices"<p>5) "Agile" is horrendously misused, the cargo cult is in full force. It should be about prioritising & doing the important things, reducing overhead to the minimum necessary. It has become an overhead in itself, with "sprints" reduced to ridiculously low periods, meeting over meeting at each sprint, etc.