Ask HN: Are we overcomplicating software development?
Many of these practices are popularized by Google/Facebook/Amazon but don't make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we're facing.Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn't a strong reason to switch to Flink or start using Haskell.Agile makes sense when your main risk is fine-grained details of user requirements, but not when you have other substantial risks, such as making sure a statistical algorithm is accurate enough.Microservices probably reduces the asymptotic cost of scaling but add a huge constant factor.Relational databases are the right choice 95% of the time, non-relational stores require a really specific use case.TDD is good for fast feedback in some domains, but for others, manually investigating the output or putting your logic into types is better. E.g. a lot of my time comes from scaling jobs that work on 10gb of data but crash on 1tb, TDD is not that helpful here.Continuous integration mostly makes sense when you're making a lot of small changes and can reliably expect a test suite to catch issues.In short, ask the question "when is practice X useful?" instead of "is practice X a good idea?"