title: Scaling Up LLM Reviews for Google Ads Content Moderation
summary: Large language models (LLMs) are powerful tools for content moderation, but
  their inference costs and latency make them prohibitive for casual use on large
  datasets, such as the Google Ads repository. This study proposes a method for
  scaling up LLM reviews for content moderation in Google Ads. First, we use
  heuristics to select candidates via filtering and duplicate removal, and create
  clusters of ads for which we select one representative ad per cluster. We then
  use LLMs to review only the representative ads. Finally, we propagate the LLM
  decisions for the representative ads back to their clusters. This method
  reduces the number of reviews by more than 3 orders of magnitude while
  achieving a 2x recall compared to a baseline non-LLM model. The success of this
  approach is a strong function of the representations used in clustering and
  label propagation; we found that cross-modal similarity representations yield
  better results than uni-modal representations.
paper_authors._id: 65d81210d1a8b32d807037a1
paper_authors.name: Wei Qiao
paper_authors._id: 65d81210d1a8b32d807037a2
paper_authors.name: Tushar Dogra
paper_authors._id: 65d81210d1a8b32d807037a3
paper_authors.name: Otilia Stretcu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-03-07T09:16:05.611Z
paper_authors._id: 65e8a55bf4335428792a9c16
paper_authors.avatarUrl: /avatars/f6c2cf3076fa8146d6c1892f65d86c05.svg
paper_authors.fullname: Otilia Stretcu
paper_authors.type: user
paper_authors.user: otiliastr
paper_authors._id: 65d81210d1a8b32d807037a4
paper_authors.name: Yu-Han Lyu
paper_authors._id: 65d81210d1a8b32d807037a5
paper_authors.name: Tiantian Fang
paper_authors._id: 65d81210d1a8b32d807037a6
paper_authors.name: Dongjin Kwon
paper_authors._id: 65d81210d1a8b32d807037a7
paper_authors.name: Chun-Ta Lu
paper_authors._id: 65d81210d1a8b32d807037a8
paper_authors.name: Enming Luo
paper_authors._id: 65d81210d1a8b32d807037a9
paper_authors.name: Yuan Wang
paper_authors._id: 65d81210d1a8b32d807037aa
paper_authors.name: Chih-Chun Chia
paper_authors._id: 65d81210d1a8b32d807037ab
paper_authors.name: Ariel Fuxman
paper_authors._id: 65d81210d1a8b32d807037ac
paper_authors.name: Fangzhou Wang
paper_authors._id: 65d81210d1a8b32d807037ad
paper_authors.name: Ranjay Krishna
paper_authors._id: 65d81210d1a8b32d807037ae
paper_authors.name: Mehmet Tek
paper_title: Scaling Up LLM Reviews for Google Ads Content Moderation
paper_summary: Large language models (LLMs) are powerful tools for content moderation, but
  their inference costs and latency make them prohibitive for casual use on large
  datasets, such as the Google Ads repository. This study proposes a method for
  scaling up LLM reviews for content moderation in Google Ads. First, we use
  heuristics to select candidates via filtering and duplicate removal, and create
  clusters of ads for which we select one representative ad per cluster. We then
  use LLMs to review only the representative ads. Finally, we propagate the LLM
  decisions for the representative ads back to their clusters. This method
  reduces the number of reviews by more than 3 orders of magnitude while
  achieving a 2x recall compared to a baseline non-LLM model. The success of this
  approach is a strong function of the representations used in clustering and
  label propagation; we found that cross-modal similarity representations yield
  better results than uni-modal representations.
paper_ai_summary: A method is proposed for scaling content moderation with large language models by clustering ads and reviewing representatives, achieving high recall with cross-modal similarity representations.
paper_ai_keywords: large language models LLMs content moderation Google Ads heuristics filtering duplicate removal clusters representative ads label propagation cross-modal similarity representations uni-modal representations