  Coming up with effective ad text is a time consuming process, and
particularly challenging for small businesses with limited advertising
experience. When an inexperienced advertiser onboards with a poorly written ad
text, the ad platform has the opportunity to detect low performing ad text, and
provide improvement suggestions. To realize this opportunity, we propose an ad
text strength indicator (TSI) which: (i) predicts the click-through-rate (CTR)
for an input ad text, (ii) fetches similar existing ads to create a
neighborhood around the input ad, (iii) and compares the predicted CTRs in the
neighborhood to declare whether the input ad is strong or weak. In addition, as
suggestions for ad text improvement, TSI shows anonymized versions of superior
ads (higher predicted CTR) in the neighborhood. For (i), we propose a BERT
based text-to-CTR model trained on impressions and clicks associated with an ad
text. For (ii), we propose a sentence-BERT based semantic-ad-similarity model
trained using weak labels from ad campaign setup data. Offline experiments
demonstrate that our BERT based text-to-CTR model achieves a significant lift
in CTR prediction AUC for cold start (new) advertisers compared to bag-of-words
based baselines. In addition, our semantic-textual-similarity model for similar
ads retrieval achieves a precision@1 of 0.93 (for retrieving ads from the same
product category); this is significantly higher compared to unsupervised
TF-IDF, word2vec, and sentence-BERT baselines. Finally, we share promising
online results from advertisers in the Yahoo (Verizon Media) ad platform where
a variant of TSI was implemented with sub-second end-to-end latency.

Computation and Language;Information Retrieval