title: Multi-line AI-assisted Code Authoring
summary: CodeCompose is an AI-assisted code authoring tool powered by large language
  models (LLMs) that provides inline suggestions to 10's of thousands of
  developers at Meta. In this paper, we present how we scaled the product from
  displaying single-line suggestions to multi-line suggestions. This evolution
  required us to overcome several unique challenges in improving the usability of
  these suggestions for developers.
    First, we discuss how multi-line suggestions can have a 'jarring' effect, as
  the LLM's suggestions constantly move around the developer's existing code,
  which would otherwise result in decreased productivity and satisfaction.
    Second, multi-line suggestions take significantly longer to generate; hence
  we present several innovative investments we made to reduce the perceived
  latency for users. These model-hosting optimizations sped up multi-line
  suggestion latency by 2.5x.
    Finally, we conduct experiments on 10's of thousands of engineers to
  understand how multi-line suggestions impact the user experience and contrast
  this with single-line suggestions. Our experiments reveal that (i) multi-line
  suggestions account for 42% of total characters accepted (despite only
  accounting for 16% for displayed suggestions) (ii) multi-line suggestions
  almost doubled the percentage of keystrokes saved for users from 9% to 17%.
  Multi-line CodeCompose has been rolled out to all engineers at Meta, and less
  than 1% of engineers have opted out of multi-line suggestions.
paper_authors._id: 65c2f4940aa2d53135b25abc
paper_authors.name: Omer Dunay
paper_authors._id: 65c2f4940aa2d53135b25abd
paper_authors.name: Daniel Cheng
paper_authors._id: 65c2f4940aa2d53135b25abe
paper_authors.name: Adam Tait
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-24T14:36:56.503Z
paper_authors._id: 629e43cba5bdf6dd379af7e7
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1654539209397-noauth.png
paper_authors.fullname: Adam Tait
paper_authors.type: user
paper_authors.user: adamtait
paper_authors._id: 65c2f4940aa2d53135b25abf
paper_authors.name: Parth Thakkar
paper_authors._id: 65c2f4940aa2d53135b25ac0
paper_authors.name: Peter C Rigby
paper_authors._id: 65c2f4940aa2d53135b25ac1
paper_authors.name: Andy Chiu
paper_authors._id: 65c2f4940aa2d53135b25ac2
paper_authors.name: Imad Ahmad
paper_authors._id: 65c2f4940aa2d53135b25ac3
paper_authors.name: Arun Ganesan
paper_authors._id: 65c2f4940aa2d53135b25ac4
paper_authors.name: Chandra Maddila
paper_authors._id: 65c2f4940aa2d53135b25ac5
paper_authors.name: Vijayaraghavan Murali
paper_authors._id: 65c2f4940aa2d53135b25ac6
paper_authors.name: Ali Tayyebi
paper_authors._id: 65c2f4940aa2d53135b25ac7
paper_authors.name: Nachiappan Nagappan
paper_title: Multi-line AI-assisted Code Authoring
paper_summary: CodeCompose is an AI-assisted code authoring tool powered by large language
  models (LLMs) that provides inline suggestions to 10's of thousands of
  developers at Meta. In this paper, we present how we scaled the product from
  displaying single-line suggestions to multi-line suggestions. This evolution
  required us to overcome several unique challenges in improving the usability of
  these suggestions for developers.
    First, we discuss how multi-line suggestions can have a 'jarring' effect, as
  the LLM's suggestions constantly move around the developer's existing code,
  which would otherwise result in decreased productivity and satisfaction.
    Second, multi-line suggestions take significantly longer to generate; hence
  we present several innovative investments we made to reduce the perceived
  latency for users. These model-hosting optimizations sped up multi-line
  suggestion latency by 2.5x.
    Finally, we conduct experiments on 10's of thousands of engineers to
  understand how multi-line suggestions impact the user experience and contrast
  this with single-line suggestions. Our experiments reveal that (i) multi-line
  suggestions account for 42% of total characters accepted (despite only
  accounting for 16% for displayed suggestions) (ii) multi-line suggestions
  almost doubled the percentage of keystrokes saved for users from 9% to 17%.
  Multi-line CodeCompose has been rolled out to all engineers at Meta, and less
  than 1% of engineers have opted out of multi-line suggestions.