Self-Crashing Cars
Real problem, clueless author.<p>The big problem is allowing firmware updates.<p>Many years ago, I knew some of the people involved with the Ford Electronic Engine Control IV, the "EEC IV", used in 1980s Fords.  This used a custom CPU, an Intel 8061, which was an Intel 8051 with some extra timer features. The program was <i>etched onto the CPU silicon</i>. There is no way to change that program without replacing the whole unit. There's an external ROM with some tables, different for each engine model, but it's a true ROM, not something that can be rewritten. If you want to replace it, it takes a wrench.<p>The level of paranoia and testing that went into that program was very high.  Any bug meant bringing back hundreds of thousands of Fords in a recall. That didn't happen, and there are still many EEC IV vehicles running today, 30 years later. That's what it was like in the days of hardcore embedded programming.<p>I'd argue that safety critical vehicle software should not be downloadable.  If a fix is needed, the vehicle has to go back to the dealer for a new memory module. This would discourage "shit early, shit often" software development, and encourage manufacturers to keep the safety critical systems, like ABS, totally disconnected from the entertainment system.<p><i>Autonomous vehicles should not communicate with each other.</i> Waymo's cars don't.  Most of the schemes for "car to car communication" are motivated by marketing or surveillance, not driving.  Aircraft systems don't communicate with the ground much, and when they do, the uploaded data is very simple.  (It's common in commercial aircraft to hardware disable maintenance functions unless the "weight on wheels" switch is on, indicating the plane is on the ground and parked.)