Practical SQL for Data Analysis
tldr ; if you hear &#x27;&#x27;but we can do this in SQL&#x27;&#x27;, RUN!!!<p>My eyes hurt as I read this article. There are reasons why analysts dont use SQL to do their job, and it has nothing to do with saving RAM and memory.<p>1) Data analysis is not a linear process, it involve playing and manipulating the data in different way and letting your mind drift a bit. You want your project in an IDE made for that purpose, the ability to create charts, source control, export and share the information, ect. Pandas is just a piece of that puzzle which is not possible to replicate in pure SQL.<p>2) In 2020, there are numerical methods you want to try beyond a traditional regression. Most real world data problems are not made for stats101 tools included in sql. Kurtosis? Autocorrelation?<p>3) Politics. Most database administrator are control freaks who <i>hate</i> the idea of somebody else doing stuff in their DB. Right now were I work we still have to use SSIS-2013 instead of stored procedures in order to avoid the DBA refusal bureaucratic process.<p>4) Eventual professional development. If your analysis is good and creates value, chances are it will become a &#x27;real&#x27; program and you will have to explain what you are doing to turn in into an OOP tool. If you have CS101, good coding in python will make this process much easier than a 3000 lines spagetti-SQL SQL script.<p>5) Data cleaning. Dealing with outliers, NAN and all that jazz really depends on the problem you try to solve. The absence of a one size fits all solutions is a good case for R&#x2F;pandas&#x2F;etc. These issue will break an SQL script in no time.<p>6) Debugging in SQL. Hahahahahahahaha<p>If you are still preocupied with the ram usage of your PC to do your project, here are two solutions which infuriate a lot of DBAs I&#x27;ve worked with.<p>A) <a href="https:&#x2F;&#x2F;www.amazon.ca&#x2F;s?k=ram&amp;__mk_fr_CA=ÅMÅŽÕÑ&amp;ref=nb_sb_noss_2" rel="nofollow">https:&#x2F;&#x2F;www.amazon.ca&#x2F;s?k=ram&amp;__mk_fr_CA=ÅMÅŽ...</a><p>B) <a href="https:&#x2F;&#x2F;aws.amazon.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;aws.amazon.com&#x2F;</a>