Rise of the machines: Programmatic set to account for 80% of online ads by 2018
According to research commissioned by the Internet Advertising Bureau (IAB), programmatic's total share of online advertising in the UK rose from 28% in 2013 to 45% in 2014 and will soar to 80% by 2018. Programmatic makes up an even larger percentage of mobile ad sales, with 64% of mobile inventory sold programatically in 2014. The explosion of video and native ad formats has been key to this shift with publishers’ efforts to push them as premium alternatives proving successful. The Globe & Mail published an extensive article about the development of the programmatic advertising industry in recent years, stating that companies such a Proctor & Gamble targeted 70-75% of their ad spending automatically last year. The industry is expected to keep growing at a quick pace, as digital billboards are used more frequently, and companies increase investment in technologies. 
Few people would notice the drab industrial building that sits on a pothole-strewn road in Secaucus, N.J. There are no signs indicating what’s inside.

And yet this factory-like property is at the centre of a growing trend in the advertising world, where ads are bought and sold by computers.

This location is one of 100 data centres owned by Equinix Inc., a Silicon Valley-based firm that processes billions of advertising deals every day through its global network of computer servers.

“The financial exchanges are what we originally built our platforms on,” Frank LoPilato, senior business development executive at Equinix, said of a similar move that saw high-speed computerized trading change the stock market. “When you look at that data, it’s all about protecting that from security threats.”

As anyone with a computer or a smartphone knows, advertising is everywhere online. How ads are bought, displayed, tracked and sold is critical to just about every player on the Internet, from giants like Facebook and Google to the smallest publisher. And that process is going through a revolution that is already having sweeping implications, from how ads are targeted to consumers, to what safeguards are needed to prevent fraud.

Advertising deals used to be made by phone, fax and e-mail. Now much of that work is being done on purpose-built digital marketplaces. Known as “programmatic advertising,” this process involves computerized systems to sell online ad space to advertisers and their agencies. It is comparable to a stock market, though instead of trading shares, these markets trade digital ad space, or impressions.

The method has been gaining speed since roughly 2009 in the United States and 2011 or 2012 in Canada. Today, roughly half of the global spending on digital display ads is automated. And the figure is growing, fast.

Google’s exchange, Adx, alone receives billions of queries – that is, bids for ad space – every day. The world’s largest advertiser, Procter & Gamble, last year targeted 70 to 75 per cent of its digital advertising spending to be automated, and Verizon Inc.’s $4.4-billion (U.S.) acquisition of AOL announced in May was driven largely by the need to get its hands on the automated ad-selling technology AOL has built.

The disruption has widespread consequences for the industry, shaking up the traditional power dynamic among its key players.

Since the global economic downturn, marketers have been under pressure to cut their budgets and to demonstrate the value of their ads. Automated trading offers cost savings, but also the potential to finally better target people who might actually want to receive their message. Media agencies have been challenged to keep up with the changes. Middlemen have attempted to cash in by handling the technology side on advertisers’ behalf. But that daisy-chain buying has also opened up the industry to fraud, misleading data and other shady practices. Meanwhile, publishers are working to guard the value of the ad space they sell and to hold on to their negotiating clout in this new world.

“A high, high component of our buys are in programmatic,” Kristi Karens of snack-foods brand Mondelez Canada said. “… Everyone needs to understand where the industry is going.”

The old way was ‘silly’

For all the Internet’s promise of a paperless world, online advertising was sold in a rather old-fashioned way for a long time.

Buyers at media agencies had to pick up the phone when they wanted to place ads on a website on behalf of their client. The final deal was often a paper transaction that involved multiple, cumbersome steps.

“The process was silly,” Andrew Casale, president and CEO of Toronto-based automated ad-seller Index Exchange, said.

As the Internet grew, the number of sites soared along with the amount of advertising. Enter advertising networks. These precursors to the current automated exchanges were created in the late 1990s but began to be widely used around 2005. They built buckets of ad space – on certain types of Web pages such as finance or lifestyle – to sell to advertisers as a convenient package.

With the networks acting as middlemen, though, it was not always clear to buyers where those ads were ending up. Automated exchanges popped up to give advertisers more control, offering auctions where buyers could bid on an audience with certain characteristics, as well as a particular website’s space.

Automated bidding now happens in a few different ways.

In “real-time bidding,” advertisers or their reps register to place ads that target a specific audience, for example a 35-year-old woman who needs diapers for her baby. They also punch in the price range they are willing to pay for that digital ad space, and a host of other parameters. That information is processed in the exchange and in fractions of a second, the auction is done and the winning bidder’s ad is placed on a website.

There are also automated auctions that happen, not instantaneously as a page loads, but ahead of time. And there are “programmatic direct” deals: Advertisers who buy a lot of ad space from a publisher can negotiate specific prices for future deals. Then, publishers can give advertisers those guaranteed prices through a digital exchange.

Marketers began to take automated advertising seriously in Canada about three years ago. Now, there are very few advertisers who are not dedicating a portion of their digital ad budgets to this type of buying, or at the very least considering doing so. Automated spending is expected to reach $1.2-billion in Canada this year, or nearly 30 per cent of total spending on digital advertising, according to research firm Magna Global. That’s double the spending just two years ago, and the firm predicts it will more than double again by 2019.

And yet, some of the people making these buying decisions – and throwing real money around – are still confused about what programmatic can and cannot do.

“There are a lot of people who know it’s important and gaining momentum, but aren’t sure exactly what it is,” Harvey Carroll, CEO of IPG Mediabrands in Canada, said.

Meanwhile, dozens of ad-tech companies are fighting for a piece of this market.

“I think the smart, savvy marketer is going to question the role of all of that tech and I think there will be significant consolidation towards models that provide real value,” added Alex LePage, a marketing executive with Los Angeles-based Rubicon Project, which operates an automated exchange as well as a buying platform for advertisers. Toronto-based buying platform Chango was one piece of that consolidation: Rubicon bought it in March for $122-million.

Waste not

Just as there is a difference between being able to compare hotels and flights online versus picking up the phone to call a travel agent, there are also pricing efficiencies in using technology to buy advertising. But cost savings are just one part of the picture.

Coming to terms with this complex technology is worth it, advertisers believe, because it brings them closer to what has always been the great potential of digital advertising: the ability to speak to the right people who might buy their products, and at the exact moment when it is least likely to be considered an annoyance.

Traditionally, brands fired messages at huge audiences based on assumptions about who was reading a particular newspaper or watching a certain TV program. The Internet was supposed to make this process more exact, and in some ways it did. But so far, it’s been mostly blunt instruments. Through “re-targeting,” advertisers look at cookies showing where you’ve been and may then pummel you with ads for, say, that pair of shoes you looked at once.

But because of the volume of data that is now being gathered through automated bidding, the picture is becoming clearer. Marketers hope to be able to bid on ad space on a page where they see a 40-year-old in Red Deer, Alta., who is in the market for a truck, for example.

“They can buy the four users that really matter … and try to find the right moment to show them the right ads,” Drew Bradstock, senior product manager for Google Inc.’s Ad Exchange, said.

Marketers can also study a campaign’s performance in real time, and tweak those campaigns based on what’s working best.

“We now have a lot more visibility as to how our campaigns are doing,” Elaine Li, digital marketing manager with Clorox Canada, said. Her team can watch, in each geographic or demographic segment of people seeing the ads, how many of them clicked through or visited the website, or signed up for a newsletter. For video, they can see how much of an ad people watched. If something is lagging, Clorox can change its campaign parameters within hours of launching.

As recently as two years ago, none of Clorox Canada’s buying was automated. Now it makes up roughly 60 per cent of the company’s digital ad spending. Ms. Li foresees it growing to at least 90 per cent soon enough.

How programmatic advertising works

A graphic explainer on the multibillion-dollar trend that’s shaking up the ad industry.

It’s much easier now for marketers to launch two ads, track the response, then put more money behind the more effective ad. This strategy, called A/B testing, is an old concept in advertising. But in traditional media environments, it was more expensive to do those tests, and the old media buying processes didn’t allow advertisers to change campaigns as quickly.

“There was a more static approach to understanding audiences in the past because you couldn’t optimize on the fly,” Mr. Carroll of IPG Mediabrands said. “… Now, because you can do things in real time, there’s a whole industry that’s been built behind having that information available. Because it has tangible commercial value now.”

More testing is happening on the publisher side too. At Index Exchange, analysts sit in front of computer screens combing through lines of data. They’re trying to figure out the absolute optimal price for each publisher to sell ad space – not just the highest price they can sell at, but the highest price that has commanded the most sales for a given piece of ad space.

“This helps the publisher understand what they can sell for. Exactly,” Paul Zovighian, an analyst at Index Exchange, said. This level of analysis – with the volume of information on each bid and sale – was hard to come by when deals were done manually.

But it’s not all a race to the bottom in prices: On exchanges, bids show up as low as a penny CPM (cost-per-mille, or per thousand people who see an ad) and as high as $40, or even more. For generalized campaigns attempting to reach many people, advertisers can get space for cheap, throwing out bids to see what sticks. But for more targeted campaigns where an advertiser wants that audience – a bank looking to speak to young couples in the market for a mortgage, for example – the bids climb higher.

“The days of [broadcasting] one 30-second or 60-second commercial to all people should be over,” Mondelez’s Ms. Karens said. “What we are delivering is more customized messaging that is relevant to every consumer, and we’re using programmatic as a tool to get there.”

Google Inc. offices at 111 Eighth Ave. in New York City. (Neville Elder for The Globe and Mail)

The fox in the henhouse

The gold lettering is still visible on the Google Inc. headquarters in Manhattan, reminding visitors that this Goliath structure taking up an entire city block was once Inland Terminal Number One for the New York Port Authority. It’s a fitting heritage, given that Google is now terminal No. 1 for so much of the money flowing into online advertising.

In this automated world, Google has its hands in everything. Its Adx, launched in 2009, is the largest exchange in the market by far. Its ad server, DoubleClick for Publishers, works for sellers; it handles the buying side as well, with its DoubleClick Bid Manager. Because of its scale, the company has been able to invest heavily to tackle some of the problems that have arisen because of computerized trading.

Those problems include ads that publishers fear will make their websites look bad (such as those warning about belly fat, or promoting “one weird trick” to reverse aging). Meanwhile, advertisers fear placing ads on sites with fake or poorly tracked audience data.

“We’re the cleanest, bar none,” Google’s Mr. Bradstock said.

That’s worth a lot of money in what can be a dirty trading environment.

Because trades are moving so fast, fraud has become worse. That’s when hackers create bots to look like humans online, luring automated ad spending to low-rent sites where criminals pocket the money. Because the fake humans need to be convincing, bot traffic visits established websites too. A study commissioned by the U.S. Association of National Advertisers last year estimated that globally, $6.3-billion (U.S.) will be spent on ads that no human ever sees.

Google is fighting another problem: high-speed arbitrage. All Google real-time auctions happen in a 100-millisecond window, so that someone with a faster connection cannot snap up an ad and turn around to resell it at a profit, siphoning more of the dollars that already fall away in the automated path between the advertiser and the publisher. Players in between – including agency trading desks, demand-side platforms, the ad exchange itself that also takes a cut, audience targeting firms, verification and fraud-prevention services, and others – take as much as 60 per cent of an advertiser’s media budget, according to a report last year from the World Federation of Advertisers. That leaves the publisher that actually provides the ad space with just 40 per cent of the price paid for it, compared with significantly more in the past.

But even when high-speed arbitrage is prevented, reselling still happens. Some of the trading desks that media agencies operate are taking an extra cut by buying ad space from publishers and reselling it to clients later. This is different than the typical media agency structure, which has been to buy media space on behalf of clients and take a commission. But as marketers have been under pressure to cut budgets, agency fees have also been squeezed, leading some to find other ways to make up the difference. Competing media agencies have criticized arbitrage.

“We want to be completely focused on client objectives,” Tessa Ohlendorf, managing director for Cadreon in Canada, said. “… We don’t want to be motivated to make money off our inventory as opposed to buying CBC or Kijiji or The Globe and Mail because it does better for the client.”

The speed and volume of automated ad-buying makes it hard to track. Marketers cannot check hundreds of thousands of transactions. “They may not even have auditors check randomly,” Brian Wieser, an industry analyst with Pivotal Research in New York, said. “It creates an environment that’s rife with activities that were not contemplated contractually, shall we say. … [Marketers] don’t necessarily have the resources to do proper audits at this scale.”

Publishers are scrambling to keep up with the changes.

“Now that the advertisers can identify the specific characteristics of the consumers they want, and find them across properties, easily, for less, the negotiating dynamic has changed radically,” Mr. Wieser said.

Some publishers have created alliances to try to protect their negotiating power. A Canadian group – including Rogers Media, Shaw Media, CBC Radio-Canada and Postmedia – have banded together to form the Canadian Premium Audience Exchange (CPAX) in an attempt to command higher prices from advertisers that want a guarantee of higher-quality ad space. Bell Media has so far opted out of the automated marketplace altogether.

The Guardian, the Financial Times, CNN International, Reuters and The Economist formed an automated selling collective called the Pangaea Alliance in March, using Rubicon’s technology. Publishers in France launched a similar group, La Place Média, in 2012. These alliances are an attempt to build up a fraction of the kind of scale that has allowed tech giants such as Google and Facebook to wrest ad spending away from smaller players.

“If you’re the publisher of a legacy title, in print and digital,” Mr. Wieser said, “you don’t have anything like the negotiating clout you used to.”

That clout is all moving in one direction: toward the tech giants that dominate the space. Some say this is not a fair fight.

Mr. Casale, for one, believes that players like Google and Rubicon have a conflict of interest.

“If you’re giving all of your data to Google, who is also trying to price intelligently, who is also a publisher, there’s a conflict there,” Mr. Casale said.

Google insists there is no problem. “The exchange is extremely neutral,” Mr. Bradstock said. “… We’ve gone out of our way to ensure that all buyers are treated equally.”

Ian Hewetson, vice president of client services for demand-side platform Eyereturn Marketing, is concerned that allowing some players to become too dominant could leave others without any leverage in the market – whatever their reassurances about neutrality.

“The fox could tell you, ‘I’m going into the hen house, and it’s going to be fine.’ Maybe it’s a very tame fox, maybe it’s a pet fox. … It’s just set up in a way that encourages trouble.”

Changing the game

It now seems inevitable that automated buying will take over the majority of the online advertising space within a few years. The next step is to automate buying in other media: Billboards that have digital displays already have the pipes to process sales this way. In the U.S. especially, there is also talk of automated buying moving into TV.

The most obvious way in is through broadcasters’ streaming services such as Rogers GameCentre Live, where viewers could be targeted with video ads attuned to their interests, said Alan Dark, senior vice-president of media sales at Rogers Media.

“In the future, we don’t know if we’re going to be selling 30-second spot units program by program,” Mr. Dark said. “We may be selling very specific audiences to marketers who don’t really care if they’re running in show X or show Y.”

There is still work to be done with online ads, too. Targeting is becoming more difficult as people spend far more time online via mobile devices. The traditional method of tracking was through cookies on laptops and computers; but on mobile, cookies often don’t exist.

It’s one reason why digital subscriptions are valuable for publishers: Log-ins offer both identity and trackable interests. “In these walled-garden environments, the publisher might know a hell of a lot more than the buyer. That can be leverage,” Mr. Casale said.

The next big thing will be to build user IDs that can identify people across devices.

The tech companies that have made billions in online advertising are envisioning a future beyond just being publishers, or sellers of ad space on behalf of others. Their big bet is on data.

“We know more about our users than any other publisher or player in the ecosystem, and we know them across devices,” Steve Irvine, global head of Facebook’s Marketing Partner Program said. Facebook wants to take the information it has about the 1.4-billion people who use its service – such as age, gender, location and interests – and track them even outside of its social network. To maximize that leverage it bought ad-measurement company Atlas from Microsoft in 2013 for almost $100-million.

The race to perfect the technology for more targeted advertising is just getting started. There are billions of dollars up for grabs for the winners. If it works, consumers could notice ads becoming even more personal. The ultimate goal is to know who you are, and what you want – not using a primitive tool like a cookie, but a much more detailed cross-device profile – and the best time to advertise to you, wherever you are.

“That will be a game changer,” Mr. Bradstock said.