Ask HN: Are we overcomplicating software development?
Many of these practices are popularized by Google&#x2F;Facebook&#x2F;Amazon but don&#x27;t make sense for a company with 100 or even 1,000 people. I try to focus on whether a practice will solve a concrete problem we&#x27;re facing.<p>Switching from Hadoop to Spark was clearly a good idea for our team, even though it required learning a new stack, but there isn&#x27;t a strong reason to switch to Flink or start using Haskell.<p>Agile makes sense when your main risk is fine-grained details of user requirements, but not when you have other substantial risks, such as making sure a statistical algorithm is accurate enough.<p>Microservices probably reduces the asymptotic cost of scaling but add a huge constant factor.<p>Relational databases are the right choice 95% of the time, non-relational stores require a really specific use case.<p>TDD is good for fast feedback in some domains, but for others, manually investigating the output or putting your logic into types is better. E.g. a lot of my time comes from scaling jobs that work on 10gb of data but crash on 1tb, TDD is not that helpful here.<p>Continuous integration mostly makes sense when you&#x27;re making a lot of small changes and can reliably expect a test suite to catch issues.<p>In short, ask the question &quot;when is practice X useful?&quot; instead of &quot;is practice X a good idea?&quot;