They don't even know the fundamentals
> It turns out my day to day work doesn’t require deep knowledge about database internals and I can mostly treat them as a black box with an API 2.<p>Because of such attitude of the previous dev team my client ended up with DB integrity ruined. Previous guys somehow didn't know they should use transactions when updating/deleting stuff in the DB, because hey, it's just API you call, who cares of mambo-jumbo happening behind the curtains, right?<p>I myself had a similar fuckup with MongoDB a decade ago, because I used it "because it's fast", but failed to read the small letters, and wasn't aware that it achieves that speed by buffering the data for async save, and so has no guarantees it will actually be saved at all. So I lost a lot of data when traffic suddenly jumped, and since those were affiliate clicks lots of people got very pissed about not getting paid, and it almost ruined both my client's business as well as my own, as it was our core client.<p>Lack of understanding of (or even caring to know about) DB internals is also why like 90% of projects I've seen has wrong indices, or often no even a single index set at all. Because, hey, it's someone else's job to think about it, but in reality budgets are limited and team don't have a dedicated DBA or even a capable devops person to notice it, and it just ends up in the production like that.<p>So, in summary: no you don't need to know every in-and-out of every technology out there, no one can do that, but there's a valid requirement that one has to be familiar to those that they actually use - at least those that can seriously bite one's ass and cost a lot of money. And all DB related stuff is very much like that, that is probably the most critical part of the whole system, so no, it's definitely not a black box.