Facebook investigating AI-making AI

Facebook is working to create an 'automated machine learning engineer' that will be able to build artificially intelligent systems on behalf of engineers. Facebook trains and tests over 300k new machine learning models every month, then uses the artificial intelligence (AI) of AutoML to refine the results to improve the training of other machine learning models, by choosing algorithms that are most likely to work. The Facebook ads team has its own automated machine learning engineer called Asimo, and LinkedIn, Uber, and Twitter have also noted their enthusiasm to use AI in the same way.

Deep neural networks are remaking the Internet. Able to learn very human tasks by analyzing vast amounts of digital data, these artificially intelligent systems are injecting online services with a power that just wasn't viable in years past. They're identifying faces in photos and recognizing commands spoken into smartphones and translating conversations from one language to another. They're even helping Google choose its search results. All this we know. But what's less discussed is how the giants of the Internet go about building these rather remarkable engines of AI.

Part of it is that companies like Google and Facebook pay top dollar for some really smart people. Only a few hundred souls on Earth have the talent and the training needed to really push the state-of-the-art forward, and paying for these top minds is a lot like paying for an NFL quarterback. That's a bottleneck in the continued progress of artificial intelligence. And it's not the only one. Even the top researchers can't build these services without trial and error on an enormous scale. To build a deep neural network that cracks the next big AI problem, researchers must first try countless options that don't work, running each one across dozens and potentially hundreds of machines.

"It's almost like being the coach rather than the player," says Demis Hassabis, co-founder of DeepMind, the Google outfit behind the history-making AI that beat the world's best Go player. "You're coaxing these things, rather than directly telling them what to do."

That's why many of these companies are now trying to automate this trial and error---or at least part of it. If you automate some of the heavily lifting, the thinking goes, you can more rapidly push the latest machine learning into the hands of rank-and-file engineers---and you can give the top minds more time to focus on bigger ideas and tougher problems. This, in turn, will accelerate the progress of AI inside the Internet apps and services that you and I use every day.

In other words, for computers to get smarter faster, computers themselves must handle even more of the grunt work. The giants of the Internet are building computing systems that can test countless machine learning algorithms on behalf of their engineers, that can cycle through so many possibilities on their own. Better yet, these companies are building AI algorithms that can help build AI algorithms. No joke. Inside Facebook, engineers have designed what they like to call an "automated machine learning engineer," an artificially intelligent system that helps create artificially intelligent systems. It's a long way from perfection. But the goal is to create new AI models using as little human grunt work as possible.

Feeling the Flow

After Facebook's $104 billion IPO in 2012, Hussein Mehanna and other engineers on the Facebook ads team felt an added pressure to improve the company's ad targeting, to more precisely match ads to the hundreds of millions of people using its social network. This meant building deep neural networks and other machine learning algorithms that could make better use of the vast amounts of data Facebook collects on the characteristics and behavior of those hundreds of millions of people.

'The more ideas you try, the better. The more data you try, the better.'

According to Mehanna, Facebook engineers had no problem generating ideas for new AI, but testing these ideas was another matter. So he and his team built a tool called Flow. "We wanted to build a machine-learning assembly line that all engineers at Facebook could use," Mehanna says. Flow is designed to help engineers build, test, and execute machine learning algorithms on a massive scale, and this includes practically any form of machine learning---a broad technology that covers all services capable of learning tasks largely on their own.

Basically, engineers could readily test an endless stream of ideas across the company's sprawling network of computer data centers. They could run all sorts of algorithmic possibilities---involving not just deep learning but other forms of AI, including logistic regression to boosted decision trees---and the results could feed still more ideas. "The more ideas you try, the better," Mehanna says. "The more data you try, the better." It also meant that engineers could readily reuse algorithms that others had built, tweaking these algorithms and applying them to other tasks.

Soon, Mehanna and his team expanded Flow for use across the entire company. Inside other teams, it could help generate algorithms that could choose the links for your Faceboook News Feed, recognize faces in photos posted to the social network, or generate audio captions for photos so that the blind can understand what's in them. It could even help the company determine what parts of the world still need access to the Internet.