Facebook continues to fail on metrics
Last week, Facebook reported flaws in its metrics used by advertisers to assess the success of ad campaigns on the platform, overstating the reach of Facebook Pages and Instant articles. The firm already admitted a few months ago that it had inflated the average time viewers spent watching video ads for over two years. While the company has promised more transparency, Wired warns that advertisers may become more sceptical about the platform, and media industry analyst Brian Wieser from Pivotal Research suggests the bigger problem is that Facebook is "grading its own homework". 

The internet was supposed to mean a whole new world for the business of advertising. Gobs of data let advertisers become wildly efficient in who they target and how they measure results. Consumers also ostensibly win: If you're in the market for a quality winter coat, the thinking goes, you're not going to be annoyed if you see an ad for one.

In this new world, Facebook is on top. It knows so much about its users that it can deliver ads precisely calibrated for virtually any demographic you can dream of, from suburban grandmothers to millennials living abroad. But lately, Facebook has faltered, exposing cracks in the basic assumptions about the superiority of digital advertising—the business model on which so much of the internet has run for the past 20 years.

Last week, Facebook said it found flaws in the metrics it reported to advertisers—the measurements by which those advertisers judge the success of their ad campaigns on the platform. The company said it overstated the reach of Facebook Pages and Instant Articles, as well as its count of referrals to apps from ads. This admission of miscounting came just a few months after Facebook said it had inflated how much time on average viewers spent watching video ads for two years.

Facebook has promised more transparency. But in media and advertising circles, some critics are starting to ask whether they've been spending their money wisely on Facebook. Were they duped into making costly business decisions based on wrong information? Yes, Facebook is still an enormously powerful platform. Close to a fifth of the world’s population checks the social network every day, and all those eyeballs are incredibly valuable to advertisers and publishers. But the revelation that it exaggerated its own reach points to real flaws in the narrative about what Facebook—and indeed, digital advertising on the whole—can accomplish.

Miscounting Metrics

Facebook says it has an incredibly robust system that offers up to 220 different ways to measure how well advertisers' ads do. But Facebook's sterling reputation has left advertisers all the more confounded by how Facebook's miscounts could have happened in the first place. “Facebook is supposed to be the best of the best, the cream of the crop, with advertisers spending good money with them,” says Brian Wieser, a media industry analyst at Pivotal Research. “There’s an expectation that you don’t get this thing wrong.”

'Facebook is supposed to be the best of the best. There’s an expectation that you don’t get this thing wrong.'

And yet Facebook did. It overstated app referrals by 6 percent on average by counting not only posts that directed traffic back to the app maker’s website or app but also clicks to view photos or video that kept a user inside of Facebook. Facebook Pages—the pages brands maintain as their home bases on the site—double-counted repeat visitors, leading to greatly exaggerated estimates of the size of their audiences. (The company said page owners should be prepared to see their "28-day reach" fall by 55 percent.) Facebook also said that, due to a math error, it estimated audiences were spending 7 to 8 percent more time reading fast-loading Instant Articles than they really were.

The company doesn’t say exactly how it came to discover these new discrepancies, pointing to “bugs” in the system. But Wieser says it’s likely that Facebook looked into the issue more deeply after finding the mistake in counting that video metric back in October. “Pick your analogy or theory,” Wieser says. “The cockroach theory, where you see one somewhere, then you see more, and say, geez, this must be something systemic. Or the Comey analogy: the problem was brought up then seemed to be settled, but then it comes up again.” Whatever the case, he says, the mistakes definitely amount to a “sackable offense.” (Facebook says it has established what it's calling a measurement council to respond to advertisers' concerns.)

But this still doesn't get at what Wieser sees as the bigger problem: Facebook "grading its own homework." You can’t get a report on Facebook metrics from anywhere else—only from Facebook. And Facebook defines what those metrics are. In the wake of its most recent debacle, the company did pledge to work with third-party industry reviewers, something Rob Norman, chief digital officer of powerhouse media buyer GroupM, applauds. But these reviewers can only retroactively verify Facebook's metrics. They can't get an independent view into the data themselves.