Machine learning’s crumbling foundations
It's a structural issue caused by the way wealth creation works for majority of people in tech. Job hopping, trendy frameworks in CV, "high-impact" projects done ASAP, etc.<p>No one wants to do boring, slow pace work with lots of planning, reflection and introspection. And why would they do it? These kind of jobs are usually worst paid. We, the practitioners, have every economic incentive to go the other route.<p>The problem goes far wider in tech than just ML. And unless the society collectively learns to appreciate patience and long-term thinking, as virtues above all else, it won't go away any time soon. What can be done is to discourage use of ML systems if an explainable deterministic system can be used (even one developed in a rush). For example credit scoring. Rules are good while black box artificial neural network isn't, even if the NN has some % more accuracy. Then if the rules are not good then can be amended and in special cases customer support could also override the rules based on human (hopefully unbiased) judgement.<p>The problem mentioned in the article of COVID-19 detection based on radiology scans is an example of a system which needs ANNs due to the nature of image processing (very difficult problem for rules AI). While techniques such as ShAP could be helpful a radiologist still needs to check because ANNs learn a lot of useless noise very often and the prediction can be nonsensical. Here it would be best to use PCR tests, serology or any more traditional and "boring" tool as it works. Luckily that is the case and shit CNN models start and end their lives in some useless paper.