title: Proofread: Fixes All Errors with One Tap
summary: The impressive capabilities in Large Language Models (LLMs) provide a
  powerful approach to reimagine users' typing experience. This paper
  demonstrates Proofread, a novel Gboard feature powered by a server-side LLM in
  Gboard, enabling seamless sentence-level and paragraph-level corrections with a
  single tap. We describe the complete system in this paper, from data
  generation, metrics design to model tuning and deployment. To obtain models
  with sufficient quality, we implement a careful data synthetic pipeline
  tailored to online use cases, design multifaceted metrics, employ a two-stage
  tuning approach to acquire the dedicated LLM for the feature: the Supervised
  Fine Tuning (SFT) for foundational quality, followed by the Reinforcement
  Learning (RL) tuning approach for targeted refinement. Specifically, we find
  sequential tuning on Rewrite and proofread tasks yields the best quality in SFT
  stage, and propose global and direct rewards in the RL tuning stage to seek
  further improvement. Extensive experiments on a human-labeled golden set showed
  our tuned PaLM2-XS model achieved 85.56\% good ratio. We launched the feature
  to Pixel 8 devices by serving the model on TPU v5 in Google Cloud, with
  thousands of daily active users. Serving latency was significantly reduced by
  quantization, bucket inference, text segmentation, and speculative decoding.
  Our demo could be seen in https://youtu.be/4ZdcuiwFU7I{Youtube}.
paper_authors._id: 66666fd2628a6ad02e5fb482
paper_authors.name: Renjie Liu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T11:39:50.606Z
paper_authors._id: 6407f6d6656adc27c7727184
paper_authors.avatarUrl: /avatars/efb7be70f28fbd58ce2ddbe3850aeb2c.svg
paper_authors.fullname: Renjie
paper_authors.type: user
paper_authors.user: renjieliu
paper_authors._id: 66666fd2628a6ad02e5fb483
paper_authors.name: Yanxiang Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T16:46:58.815Z
paper_authors._id: 6506345fd8c09885d10182b9
paper_authors.avatarUrl: /avatars/0d9b4dd639a6f9796a17a9c0bbf222da.svg
paper_authors.fullname: Winter Zhang
paper_authors.type: user
paper_authors.user: stdcoutzyx
paper_authors._id: 66666fd2628a6ad02e5fb484
paper_authors.name: Yun Zhu
paper_authors._id: 66666fd2628a6ad02e5fb485
paper_authors.name: Haicheng Sun
paper_authors._id: 66666fd2628a6ad02e5fb486
paper_authors.name: Yuanbo Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T11:40:41.346Z
paper_authors._id: 64d33b19e185b93d6b077c15
paper_authors.avatarUrl: /avatars/e5db4cb4ed959c4463d83b01383a8ace.svg
paper_authors.fullname: Yuanbo Zhang
paper_authors.type: user
paper_authors.user: Zyb4ever
paper_authors._id: 66666fd2628a6ad02e5fb487
paper_authors.name: Michael Xuelin Huang
paper_authors._id: 66666fd2628a6ad02e5fb488
paper_authors.name: Shanqing Cai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-06-10T11:41:12.437Z
paper_authors._id: 63a5dfba28af3c9aa20703d7
paper_authors.avatarUrl: /avatars/a47f248d01775a3ebf5819dd9367a761.svg
paper_authors.fullname: Shanqing Cai
paper_authors.type: user
paper_authors.user: sqcai
paper_authors._id: 66666fd2628a6ad02e5fb489
paper_authors.name: Lei Meng
paper_authors._id: 66666fd2628a6ad02e5fb48a
paper_authors.name: Shumin Zhai
paper_title: Proofread: Fixes All Errors with One Tap
paper_summary: The impressive capabilities in Large Language Models (LLMs) provide a
  powerful approach to reimagine users' typing experience. This paper
  demonstrates Proofread, a novel Gboard feature powered by a server-side LLM in
  Gboard, enabling seamless sentence-level and paragraph-level corrections with a
  single tap. We describe the complete system in this paper, from data
  generation, metrics design to model tuning and deployment. To obtain models
  with sufficient quality, we implement a careful data synthetic pipeline
  tailored to online use cases, design multifaceted metrics, employ a two-stage
  tuning approach to acquire the dedicated LLM for the feature: the Supervised
  Fine Tuning (SFT) for foundational quality, followed by the Reinforcement
  Learning (RL) tuning approach for targeted refinement. Specifically, we find
  sequential tuning on Rewrite and proofread tasks yields the best quality in SFT
  stage, and propose global and direct rewards in the RL tuning stage to seek
  further improvement. Extensive experiments on a human-labeled golden set showed
  our tuned PaLM2-XS model achieved 85.56\% good ratio. We launched the feature
  to Pixel 8 devices by serving the model on TPU v5 in Google Cloud, with
  thousands of daily active users. Serving latency was significantly reduced by
  quantization, bucket inference, text segmentation, and speculative decoding.
  Our demo could be seen in https://youtu.be/4ZdcuiwFU7I{Youtube}.
paper_ai_keywords: Large Language Models (LLMs) Proofread Gboard server-side LLM sentence-level corrections paragraph-level corrections data generation metrics design model tuning deployment data synthetic pipeline multifaceted metrics Supervised Fine Tuning (SFT) Reinforcement Learning (RL) Rewrite tasks proofread tasks global rewards direct rewards PaLM2-XS model TPU v5 quantization bucket inference text segmentation speculative decoding