Why is machine learning ‘hard’?
I think the problem is that we don't really understand ML properly yet.Like picking hyperparamters - time and time again I've asked experts/trainers/colleagues: "How do I know what type of model to use?  How many layers? How many nodes per layer? Dropout or not?" etc etc  And the answer is always along the lines of "just try a load of stuff and pick the one that works best".To me, that feels weird and worrying.  Its like we don't yet understand ML properly yet to definitively say, for a given data set, what sort of model we'll need.This can lead us down the debugging black-hole TFA talks about since we appear to have zero-clue about why we chose something, so debugging something ultimately might just be "opps - we chose 3 layers of 10, 15, and 11 nodes, instead of 3 layers of 10, 15 and 12 nodes!  D'oh!  Lets start training again!"It really grates me to think about this considering how much maths and proofs and algorithms get thrown at you when being taught ML, then to be told when it comes to actually doing something its all down to "intuition" (guessing).And yeah as others have said - data :-)