Minecraft pig wrangling helps AIs learn to cooperate
A team from Oxford University has won an AI competition by using reinforcement learning to get AI agents to work together in order to catch pigs in Minecraft. The Malmo Collaborative AI Challenge took place inside a special version of Minecraft, called Project Malmo, designed specifically for training machine intelligences. The team's reinforcement learning approach derives from the way animals learn by experimentation. The top teams in the contest, which is intended to promote social intelligence and cooperation between AIs, win a $20,000 research grant and a place on Microsoft's Research AI summer school.
The project also hints at how humans and AI systems might eventually work together to achieve more than the sum of their parts. “This is part of a broader trend of rethinking AI as augmented intelligence rather than artificial intelligence,” says Oren Etzioni, CEO of the Allen Institute for Artificial Intelligence.

For the Microsoft contest, AI agents worked together inside Project Malmo, a special version of the open-ended computer game Minecraft. Microsoft’s researcher designed this environment to make it straightforward to import and test different AI techniques. Much further progress will be needed before AI agents can team up in useful ways or assist humans, but the contest offers a way to test some early ideas.

For the competition, agents could try to control and catch an unruly virtual pig either on their own or by teaming up with another AI agent, earning points each time.

The top teams in the Malmo Collaborative AI Challenge used cutting-edge machine-learning approaches such as deep learning to train their agents to work together. This entailed feeding them large amounts of data. But some participants also made use of older, less fashionable approaches that involve give a virtual agent hard-coded knowledge and understanding.

The winners of the contest, a team from the University of Oxford in the U.K., used reinforcement learning, a kind of machine learning inspired by the way animals learn through experimentation (see “10 Breakthrough Technologies: Reinforcement Learning”). Their agents experienced positive reinforcement whenever they successfully worked together to grab the pig.

Katja Hofmann, the lead researcher on Microsoft’s Malmo project, notes that many teams combined different approaches. “There was no single type of approach that emerged as a clear winner,” she adds, saying it’s likely that hybrid approaches “will prove particularly promising directions for future research.”

The pig-wrestling challenge takes inspiration from a thought experiment known as the Stag Hunt, which explores concepts within game theory, a branch of mathematics concerned with cooperation and negotiation strategies. The idea is that two hunters must decide whether to hunt a hare on their own or team up to snag the bigger prize of a stag.

The top teams involved in the contest, judged according to the score they achieved as well as the novelty of their work, will receive a $20,000 research grant and a place at Microsoft’s Research AI Summer School.