Hamilton: A Microframework for Dataframe Generation
It always amazes me how readily people assume that their problem is unique and special. This article can be summarized as: standard software engineering problem gets solved with standard software engineering practices. The problem they describe of a giant function that gets touched every time something changes is called a &quot;God Function&quot; or a &quot;God Class&quot; and it&#x27;s one of the most common problems in software engineering. Their solution is called &quot;Dependency Inversion&quot; and it was designed to solve literally that exact problem in the exact way they&#x27;re doing it.<p>That doesn&#x27;t mean it&#x27;s easy! Anyone who applies Software Engineering principles to make their code better deserves congratulations. So congratulations, Stitchfix! But you could have found this solution a lot quicker if you didn&#x27;t fall into that age old trap of thinking your data-science code was <i>different</i>, and <i>excluded</i> from standard software engineering. Everyone does this, and I don&#x27;t know why. They read all these blogs and books on software engineering best practices, but then they don&#x27;t apply them to anything. You can&#x27;t apply it to your SQL because it&#x27;s just <i>different</i>. Your database organization is <i>different</i>. Your I&#x2F;O code is <i>different</i>. Your API surface is <i>different</i>. Your UI is <i>different</i>. Your feature engineering is <i>different</i>.<p>No! It&#x27;s software. Stop thinking that you need a unique solution to your unique snowflake problem. Apply well-known and good software engineering practices to everything you ever do.<p>They even say in the article &quot;It hasn’t grown convoluted out of ... bad software engineering practices&quot;. <i>Of course it has!</i> What do you think software engineering practices are supposed to apply to!?