The myth of “joins don't scale”
I get the impression that (crucially: outside of Facebook&#x2F;Google-scale applications) the strongest advocates for No-SQL databases are great software devs who just had a poor experience with SQL databases as juniors or students - or didn’t develop have the necessary experience and skills to make the most of a SQL RDBMS because it was out of their field (e.g. game devs); they’d be missing things like not understanding relational-algebra or relational-calculus - or not even knowing SQL at all - or having the DBA skills to know how to define the best indexes - or even what a cardinality-estimate in an execution plan is. If you throw all of those topics at someone and also add that they only have a week to build the thing - it’s no-wonder that people turn to No-SQL.<p>I’m not putting those people down either - I would have opted for No-SQL myself if I wasn’t already familiar with RDBS theory and management. What I am criticising is the large amount of manual-management needed when you do run an RDBMS because this is putting people off. Instead, things like automatic index generation and pruning, nightly index rebuilds and statistics computing, etc arguably SHOULD be part of the RDBS out-of-the-box and enabled by default. To my knowledge only IaaS databases have features like that (it’s what justifies their subscription pricing). I believe some on-prem database server products will generate recommendations but they won’t automatically implement them on their own accord.<p>Another “approachability” issue I have is with the design of the SQL language: it’s unnecessarily verbose and very confusing for beginners as its basis in relational-calculus instead of relational-algebra throws people off (“SELECT is evaluated <i>after</i> the WHERE clause?!”).<p>If SQL is like SGML and XML, which was replaced in many applications by the faster and lighter JSON format, can we do the same for SQL? Unfortunately there’s too much institutional inertia behind SQL for any of the DBMS vendors to go-it-alone, and beyond trivial queries all of the SQL implementations are incompatible with each other anyway.<p>In summary:<p>1. RDBMS servers should have an “automatic DBA”-mode enabled by default (because if it’s not the default then the exact people who need it (unsophisticated DB devs) won’t be using it, and the people who don’t want it will just turn it off anyway.<p>2. SQL needs to be replaced with something better: less verbose but just as expressive, and with all of SQL’s “gotchas” removed (like adding multiplicity assertions to joins, implicit joins when dereferencing a foreign-key, implicit locks, allowing comparisons to NULL, and leaning more towards relational-algebra instead of relational-calculus, and support for column macros: this is desperately needed for CRUD applications!)<p>3. Thank you for coming to my TED Talk.