MIT research leading to more transparent AI decision-making 
Researchers at MIT are investigating ways of making artificial neural networks’ decision-making more transparent. At present, artificial neural networks are not explicitly designed to reveal what factors prompted them to make a certain decision. In a recent paper called “Rationalising Neural Predictions” MIT researchers designed a neural network that would be forced to explain why it reached a certain conclusion. In one unpublished work, they used the technique to identify explanatory phrases from several thousand biopsy reports. It provides a starting point for equipping neural networks with a higher degree of accountability for their decisions.

If the Halloween month has you feeling a bit puzzled and uncertain, maybe it's because of the unsettling proposition that life-and-death decisions are increasingly placed in the hands of artificial intelligence. No, this isn't in reference to doomsday military drones developed in top-secret government labs, but rather the far more pedestrian prospect of self-driving cars and robotic surgeons. Amidst the uproar about potential job losses on account of said automation, it’s sometimes forgotten that these artificial agents will be deciding not merely who receives a paycheck, but also the question of who lives and who dies.

Fortunately for us, these thorny ethical questions have not be lost upon, say, the engineers at Ford, Tesla, and Mercedes, who are increasingly wrestling with ethics as much as efficiency and speed. For instance, should a self-driving car swerve wildly to avoid two toddlers chasing a ball into an intersection, thus endangering the driver and passengers, or continue on a collision course with the children? These types of questions are not easy, even for humans. But the difficulty is compounded when they involve artificial neural networks.

Towards this end, researchers at MIT are investigating ways of making artificial neural networks more transparent in their decision-making. As they stand now, artificial neural networks are a wonderful tool for discerning patterns and making predictions. But they also have the drawback of not being terribly transparent. The beauty of an artificial neural network is its ability to sift through heaps of data and find structure within the noise. This is not dissimilar from the way we might look up at clouds and see faces amidst their patterns. And just as we might have trouble explaining to someone why a face jumped out at us from the wispy trails of a cirrus cloud formation, artificial neural networks are not explicitly designed to reveal what particular elements of the data prompted them to decide a certain pattern was at work and make predictions based upon it.

In this beer classification example from their paper "Rationalizing Neural Predictions," the algorithm uses highlighted phrases to justify a certain conclusions it reached about a beer.

To those endowed with a innate trust of technology, this might not seem like such a terrible problem, so long as the algorithm was achieving a high level of accuracy. But we tend to want a little more explanation when human lives hang in the balance -- for instance, if an artificial neural net has just diagnosed someone with a life-threatening form of cancer and recommends a dangerous procedure. At that point, we would likely want to know what features of the person’s medical workup tipped the algorithm in favor of its diagnosis.

That's where the latest research comes in. In a recent paper called “Rationalizing Neural Predictions(Opens in a new window),” MIT researchers Lei, Barzilay, and Jaakkola designed a neural network that would be forced to provide explanations for why it reached a certain conclusion. In one unpublished work, they used the technique to identify and extract explanatory phrases from several thousand breast biopsy reports. The MIT team's method was limited to text-based analysis, and therefore significantly more intuitive than say, an image based classification system. But it nonetheless provides a starting point for equipping neural networks with a higher degree of accountability for their decisions.

Now read: Artificial neural networks are changing the world. What are they?