Hackers subtle data changes confuse military vehicles AI

Hackers could subtly change military AI data to confuse equipment, with a range of potential outcomes including steering guided missiles towards the wrong targets and using algorithmic camouflage to hide hardware from AI-connected surveillance. After a period of low concern about adversarial machine learning, despite the rise in military uses for AI, the Pentagon is reportedly beginning to take notice of the potential problems. As a demonstration, Tencent has already fooled the AI algorithms on a Tesla car, showing that deep learning algorithms are easy to fool by hackers who know their weak points.

The ambition to build the smartest, and deadliest, weapons is understandable, but as the Tesla hack shows, an enemy that knows how an AI algorithm works could render it useless or even turn it against its owners. The secret to winning the AI wars might rest not in making the most impressive weapons but in mastering the disquieting treachery of the software.

Battle bots

On a bright and sunny day last summer in Washington, DC, Michael Kanaan was sitting in the Pentagon’s cafeteria, eating a sandwich and marveling over a powerful new set of machine-learning algorithms.

A few weeks earlier, Kanaan had watched a video game in which five AI algorithms worked together to very nearly outmaneuver, outgun, and outwit five humans in a contest that involved controlling forces, encampments, and resources across a complex, sprawling battlefield. The brow beneath Kanaan’s cropped blond hair was furrowed as he described the action, though. It was one of the most impressive demonstrations of AI strategy he’d ever seen, an unexpected development akin to AI advances in chess, Atari, and other games.

The war game had taken place within Dota 2, a popular sci-fi video game that is incredibly challenging for computers. Teams must defend their territory while attacking their opponents’ encampments in an environment that is more complex and deceptive than any board game. Players can see only a small part of the whole picture, and it can take about half an hour to determine if a strategy is a winning one.

The AI combatants were developed not by the military but by OpenAI, a company created by Silicon Valley bigwigs including Elon Musk and Sam Altman to do fundamental AI research. The company’s algorithmic warriors, known as the OpenAI Five, worked out their own winning strategies through relentless practice, and by responding with moves that proved most advantageous.

AI-guided missiles could be blinded by adversarial data, and perhaps even steered back toward friendly targets.

It is exactly the type of software that intrigues Kanaan, one of the people tasked with using artificial intelligence to modernize the US military. To him, it shows what the military stands to gain by enlisting the help of the world’s best AI researchers. But whether they are willing is increasingly in question.

Kanaan was the Air Force lead on Project Maven, a military initiative aimed at using AI to automate the identification of objects in aerial imagery. Google was a contractor on Maven, and when other Google employees found that out, in 2018, the company decided to abandon the project. It subsequently devised an AI code of conduct saying Google would not use its AI to develop “weapons or other technologies whose principal purpose or implementation is to cause or directly facilitate injury to people.”