This article was taken from the November 2012 issue of Wired
magazine. Be the first to read Wired's articles in print before
they're posted online, and get your hands on loads of additional
content by subscribing online. In 2005, Deb Roy, director of the
Cognitive Machines group at MIT's Media Lab, installed 11
cameras and 14 microphones in his Boston home to study how his
newborn son learned language. The result was more than 240,000
hours of multitrack audio and video that proved incredibly hard
to decode. "The cameras had fish-eye lenses," says Roy, 44.
"When someone walked in a straight line, their image would walk
out of one warped image into another. It was extremely difficult
to make sense of." The solution, which came to Roy's team "after
struggling for years", was to use software to unwarp the images
back into linear space. The pixels were then overlaid on to a 3D
virtual model of Roy's house, making the data look real again. A
"camera", whose output resembles the point-of-view in a first-
person shoo