There is no data engineering roadmap
Author of the post here, I had no idea it got posted here and just noticed a huge traffic spike. So, hello!<p>Something I got asked a lot in response to this post, or maybe berated for, is that there is a lot more to Data Engineering than SQL.<p>FWIW, I agree. There is so, so much more to DE than SQL.<p>The point of this post was purely to cover entry-level DE, and not &#x27;This is the only thing you&#x27;ll ever need in your career&#x27;. In my view, developing your SQL skills teaches you a lot of the fundamentals of working with data, enough that you can start to use it in a professional setting. From there, you will develop a lot more skills than just SQL, but your journey could take you in so many different paths that I find it hard to agree with folks who say you should always learn X,Y,Z specific tools. Some folks will end up in Python shops, or Scala, or Bash, or no-code, Airflow, Dagster, Prefect, Control-M, Spark, Flank, Pandas, Polrs, BigQuery, Snowflake, Redshift, ClickHouse, MSSQL, Hadoop, dbt....Some might even go down the K8S and Terraform road, becoming more of a platform-focused DE...<p>There is just so much potential variance depending on where you work and who you are, that it doesn&#x27;t sit well with me to shoe-horn everyone into a pattern of &quot;First learn Airflow, then learn BigQuery, then learn Spark&quot; when you could have a successful career and never touch a single one of those tools.<p>So, the reason I focused on SQL here, is that in my career consulting with hundreds of DE teams - the only consistent skill across every single team has been SQL. I do not believe SQL is all you&#x27;ll ever need, but I do think its all you need to get started. It&#x27;s the only skill I see being relavent regardless of which team you join, and that will still be relevant in 15+ years time.