title: Why Personalizing Deep Learning-Based Code Completion Tools Matters
summary: Deep learning (DL)-based code completion tools have transformed software
  development by enabling advanced code generation. These tools leverage models
  trained on vast amounts of code from numerous repositories, capturing general
  coding patterns. However, the impact of fine-tuning these models for specific
  organizations or developers to boost their performance on such subjects remains
  unexplored. In this work, we fill this gap by presenting solid empirical
  evidence answering this question. More specifically, we consider 136 developers
  from two organizations (Apache and Spring), two model architectures (T5 and
  Code Llama), and three model sizes (60M, 750M, and 7B trainable parameters). T5
  models (60M, 750M) were pre-trained and fine-tuned on over 2,000 open-source
  projects, excluding the subject organizations' data, and compared against
  versions fine-tuned on organization- and developer-specific datasets. For the
  Code Llama model (7B), we compared the performance of the already pre-trained
  model publicly available online with the same model fine-tuned via
  parameter-efficient fine-tuning on organization- and developer-specific
  datasets. Our results show that there is a boost in prediction capabilities
  provided by both an organization-specific and a developer-specific additional
  fine-tuning, with the former being particularly performant. Such a finding
  generalizes across (i) the two subject organizations (i.e., Apache and Spring)
  and (ii) models of completely different magnitude (from 60M to 7B trainable
  parameters). Finally, we show that DL models fine-tuned on an
  organization-specific dataset achieve the same completion performance of
  pre-trained code models used out of the box and being sim10times larger,
  with consequent savings in terms of deployment and inference cost (e.g.,
  smaller GPUs needed).
paper_authors._id: 67ddc4f45c0bd58dff86341d
paper_authors.name: Alessandro Giagnorio
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-21T19:59:32.283Z
paper_authors._id: 663486a1f64712540644cb68
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/663486a1f64712540644cb68/YZFR41ERY6UrC6rCC6Nan.jpeg
paper_authors.fullname: Alessandro
paper_authors.type: user
paper_authors.user: Devy1
paper_authors._id: 67ddc4f45c0bd58dff86341e
paper_authors.name: Alberto Martin-Lopez
paper_authors._id: 67ddc4f45c0bd58dff86341f
paper_authors.name: Gabriele Bavota
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-05-06T13:41:49.887Z
paper_authors._id: 6638bea59e57161faac814e7
paper_authors.avatarUrl: /avatars/91375b88945af50e51b7229a789a31b8.svg
paper_authors.fullname: Gabriele Bavota
paper_authors.type: user
paper_authors.user: gbavota
paper_title: Why Personalizing Deep Learning-Based Code Completion Tools Matters
paper_summary: Deep learning (DL)-based code completion tools have transformed software
  development by enabling advanced code generation. These tools leverage models
  trained on vast amounts of code from numerous repositories, capturing general
  coding patterns. However, the impact of fine-tuning these models for specific
  organizations or developers to boost their performance on such subjects remains
  unexplored. In this work, we fill this gap by presenting solid empirical
  evidence answering this question. More specifically, we consider 136 developers
  from two organizations (Apache and Spring), two model architectures (T5 and
  Code Llama), and three model sizes (60M, 750M, and 7B trainable parameters). T5
  models (60M, 750M) were pre-trained and fine-tuned on over 2,000 open-source
  projects, excluding the subject organizations' data, and compared against
  versions fine-tuned on organization- and developer-specific datasets. For the
  Code Llama model (7B), we compared the performance of the already pre-trained
  model publicly available online with the same model fine-tuned via
  parameter-efficient fine-tuning on organization- and developer-specific
  datasets. Our results show that there is a boost in prediction capabilities
  provided by both an organization-specific and a developer-specific additional
  fine-tuning, with the former being particularly performant. Such a finding
  generalizes across (i) the two subject organizations (i.e., Apache and Spring)
  and (ii) models of completely different magnitude (from 60M to 7B trainable
  parameters). Finally, we show that DL models fine-tuned on an
  organization-specific dataset achieve the same completion performance of
  pre-trained code models used out of the box and being sim10times larger,
  with consequent savings in terms of deployment and inference cost (e.g.,
  smaller GPUs needed).
paper_ai_summary: Organizational and developer-specific fine-tuning of deep learning models for code completion enhances prediction capabilities and reduces deployment costs, generalizing across organizations and model sizes.
paper_ai_keywords: T5 Code Llama pre-trained fine-tuned parameter-efficient fine-tuning prediction capabilities deployment inference cost