The studies measured the individual IQs of each of the participants, and found that group intelligence was only moderately correlated with the average and maximum intelligence of the individual group members. But, they did find three group attributes that were statistically significant factors in predicting how well each group will do on a wide range of tasks. One was the average social sensitivity and perceptiveness of the group, that is, the ability of group members to read each other’s emotions. They also found that groups in which a few people dominated the conversation did not perform as well as those groups where speaking and contributions were more evenly distributed. Finally, the studies found that collective intelligence positively correlated with the proportion of women in the group, most likely because women generally score higher in social perceptiveness tests.
Much additional research is needed, but there seems to be evidence that something like collective intelligence does indeed exist and can be measured. It’s primarily dependent on how well the individual members of the group work together and, to a lesser extent, on their individual abilities.
In another set of studies, Malone and collaborators looked at whether groups that included both humans and computers did better at making decisions than either the humans or computers by themselves. Their experiment, reported in this working paper, used the concept of prediction markets to predict what the next play would be in a football game. Some of the predictions were made by groups of humans; some by different computer-based statistical models; and some by combining the human and computer predictions.
Malone’s talk, perhaps because it took place as the baseball playoffs were just getting underway, reminded me of the discussions on predicting the future performance of baseball players in Nate Silver’s The Signal and the Noise: Why Most Predictions Fail but Some Don’t. The book, published about a year ago, offers some of the best explanations I’ve read of the state-of-the-art of data-driven predictions in a variety of fields.
Silver gained national attention when he correctly predicted the results of the 2008 Democratic Party presidential primaries in his FiveThirtyEight website launched earlier that year. His final forecasts for the 2008 presidential elections predicted the winner in 49 of the 50 states, as well as the winner of every race in the Senate. His notoriety went even higher during the 2012 presidential elections, when he correctly predicted the winner in all 50 states, including all nine highly contested swing states, as well as in 31 of the 33 Senate races.
Silver learned his craft in the new field of sabermetrics, - the use of statistics in baseball to project a player’s performance and career. Sabermetrics was popularized by Michael Lewis in Moneyball, his bestseller book, - later turned into a film, - about Billy Beane, the Oakland Athletics general manager who used such statistical techniques to make his small-market team highly competitive against teams with much larger budgets. The Oakland A’s made the playoffs again this year, having won the American League West Division.
The Signal and the Noise devotes a whole chapter to sabermetrics. Sabermetrics is now widely used by every team in baseball. When Moneyball first came out in 2003, many viewed it as a story about the conflict between the traditional approach of the scouts, - the professional talent evaluators who learn about the players first-hand by meeting them in person and watching them play, - versus the new approaches being introduced by the statheads, - who mostly rely on sophisticated statistical analysis to predict future performance.
Years later, there was enough data to compare the performance of scouts versus more purely statistical approaches. The scouts’ predictions were about 15 percent better than those that relied on statistics alone. The good scouts, as it turns out, use a hybrid approach combining statistics with whatever else they learn about the players. Statistics alone cannot tell you everything you want to know about a player, and the additional personal evaluations of the scouts make a significant difference.
Billy Bean attributes the success of the Oakland A’s not just to their statistical aptitude but to their careful scouting of amateur players. Scouting is particularly valuable when evaluating young amateur players while they are still learning the game, because a good scout can spot certain intangibles that may make a player stand out, such as their mental makeup and overall attitude toward the game. He defines a good scout as someone who can find out information that other people can’t, such as getting to know the prospect and his family in person. In fact, their scouting budget is now much higher than it has ever been as a way of complementing their statistical analysis.
The promise of cognitive systems and data science is that combining advanced technologies and lots of data with talented individuals and groups will help us better address the highly complex problems we face in business, government and society in general. The technologies will both help enhance the performance of individuals, as well as help groups work together more effectively. This combination of people and computers should hopefully allow us to take on some of our most difficult problems in ways that had never been possible before.