title: CodeTF: One-stop Transformer Library for State-of-the-art Code LLM
summary: Code intelligence plays a key role in transforming modern software
  engineering. Recently, deep learning-based models, especially Transformer-based
  large language models (LLMs), have demonstrated remarkable potential in
  tackling these tasks by leveraging massive open-source code data and
  programming language features. However, the development and deployment of such
  models often require expertise in both machine learning and software
  engineering, creating a barrier for the model adoption. In this paper, we
  present CodeTF, an open-source Transformer-based library for state-of-the-art
  Code LLMs and code intelligence. Following the principles of modular design and
  extensible framework, we design CodeTF with a unified interface to enable rapid
  access and development across different types of models, datasets and tasks.
  Our library supports a collection of pretrained Code LLM models and popular
  code benchmarks, including a standardized interface to train and serve code
  LLMs efficiently, and data features such as language-specific parsers and
  utility functions for extracting code attributes. In this paper, we describe
  the design principles, the architecture, key modules and components, and
  compare with other related library tools. Finally, we hope CodeTF is able to
  bridge the gap between machine learning/generative AI and software engineering,
  providing a comprehensive open-source solution for developers, researchers, and
  practitioners.
paper_authors._id: 64795320bb9a5693c48c2089
paper_authors.name: Nghi D. Q. Bui
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2023-06-05T10:07:48.667Z
paper_authors._id: 63a44141de134926a2ba0458
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/63a44141de134926a2ba0458/FP3uxpsKL1nBrrKyiFne3.jpeg
paper_authors.fullname: Nghi Bui
paper_authors.type: user
paper_authors.user: bdqnghi
paper_authors._id: 64795320bb9a5693c48c208a
paper_authors.name: Hung Le
paper_authors._id: 64795320bb9a5693c48c208b
paper_authors.name: Yue Wang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-05T11:01:42.625Z
paper_authors._id: 614d32bfb06484609bf2292c
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1632449169691-noauth.png
paper_authors.fullname: Yue Wang
paper_authors.type: user
paper_authors.user: yuewang-sf
paper_authors._id: 64795320bb9a5693c48c208c
paper_authors.name: Junnan Li
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-06-05T11:01:17.826Z
paper_authors._id: 61f9d3b54ac99e8a1bae85f4
paper_authors.avatarUrl: /avatars/ac47d13204dd22452e4bc46e280842d5.svg
paper_authors.fullname: JunnanLi
paper_authors.type: user
paper_authors.user: JunnanLi
paper_authors._id: 64795320bb9a5693c48c208d
paper_authors.name: Akhilesh Deepak Gotmare
paper_authors._id: 64795320bb9a5693c48c208e
paper_authors.name: Steven C. H. Hoi
paper_title: CodeTF: One-stop Transformer Library for State-of-the-art Code LLM
paper_summary: Code intelligence plays a key role in transforming modern software
  engineering. Recently, deep learning-based models, especially Transformer-based
  large language models (LLMs), have demonstrated remarkable potential in
  tackling these tasks by leveraging massive open-source code data and
  programming language features. However, the development and deployment of such
  models often require expertise in both machine learning and software
  engineering, creating a barrier for the model adoption. In this paper, we
  present CodeTF, an open-source Transformer-based library for state-of-the-art
  Code LLMs and code intelligence. Following the principles of modular design and
  extensible framework, we design CodeTF with a unified interface to enable rapid
  access and development across different types of models, datasets and tasks.
  Our library supports a collection of pretrained Code LLM models and popular
  code benchmarks, including a standardized interface to train and serve code
  LLMs efficiently, and data features such as language-specific parsers and
  utility functions for extracting code attributes. In this paper, we describe
  the design principles, the architecture, key modules and components, and
  compare with other related library tools. Finally, we hope CodeTF is able to
  bridge the gap between machine learning/generative AI and software engineering,
  providing a comprehensive open-source solution for developers, researchers, and
  practitioners.
paper_ai_keywords: Transformer-based large language models pre-trained Code LLM models code benchmarks language-specific parsers utility functions