CodeTF: One-stop Transformer Library for State-of-the-art Code LLM
Code intelligence plays a key role in transforming modern software
engineering. Recently, deep learning-based models, especially Transformer-based
large language models (LLMs), have demonstrated remarkable potential in
tackling these tasks by leveraging massive open-source code data and
programming language features. However, the development and deployment of such
models often require expertise in both machine learning and software
engineering, creating a barrier for the model adoption. In this paper, we
present CodeTF, an open-source Transformer-based library for state-of-the-art
Code LLMs and code intelligence. Following the principles of modular design and
extensible framework, we design CodeTF with a unified interface to enable rapid
access and development across different types of models, datasets and tasks.
Our library supports a collection of pretrained Code LLM models and popular
code benchmarks, including a standardized interface to train and serve code
LLMs efficiently, and data features such as language-specific parsers and
utility functions for extracting code attributes. In this paper, we describe
the design principles, the architecture, key modules and components, and
compare with other related library tools. Finally, we hope CodeTF is able to
bridge the gap between machine learning/generative AI and software engineering,
providing a comprehensive open-source solution for developers, researchers, and
practitioners.
64795320bb9a5693c48c2089
Nghi D. Q. Bui
claimed_verified
2023-06-05T10:07:48.667Z
63a44141de134926a2ba0458
https://cdn-avatars.huggingface.co/v1/production/uploads/63a44141de134926a2ba0458/FP3uxpsKL1nBrrKyiFne3.jpeg
Nghi Bui
user
bdqnghi
64795320bb9a5693c48c208a
Hung Le
64795320bb9a5693c48c208b
Yue Wang
admin_assigned
2023-06-05T11:01:42.625Z
614d32bfb06484609bf2292c
https://cdn-avatars.huggingface.co/v1/production/uploads/1632449169691-noauth.png
Yue Wang
user
yuewang-sf
64795320bb9a5693c48c208c
Junnan Li
admin_assigned
2023-06-05T11:01:17.826Z
61f9d3b54ac99e8a1bae85f4
/avatars/ac47d13204dd22452e4bc46e280842d5.svg
JunnanLi
user
JunnanLi
64795320bb9a5693c48c208d
Akhilesh Deepak Gotmare
64795320bb9a5693c48c208e
Steven C. H. Hoi
CodeTF: One-stop Transformer Library for State-of-the-art Code LLM
Code intelligence plays a key role in transforming modern software
engineering. Recently, deep learning-based models, especially Transformer-based
large language models (LLMs), have demonstrated remarkable potential in
tackling these tasks by leveraging massive open-source code data and
programming language features. However, the development and deployment of such
models often require expertise in both machine learning and software
engineering, creating a barrier for the model adoption. In this paper, we
present CodeTF, an open-source Transformer-based library for state-of-the-art
Code LLMs and code intelligence. Following the principles of modular design and
extensible framework, we design CodeTF with a unified interface to enable rapid
access and development across different types of models, datasets and tasks.
Our library supports a collection of pretrained Code LLM models and popular
code benchmarks, including a standardized interface to train and serve code
LLMs efficiently, and data features such as language-specific parsers and
utility functions for extracting code attributes. In this paper, we describe
the design principles, the architecture, key modules and components, and
compare with other related library tools. Finally, we hope CodeTF is able to
bridge the gap between machine learning/generative AI and software engineering,
providing a comprehensive open-source solution for developers, researchers, and
practitioners.
CodeTF is an open-source library that facilitates the use and development of Transformer-based large language models for code intelligence, offering a unified interface and support for various pre-trained models and benchmarks.
Transformer-based large language models (LLMs) Code LLMs modular design extensible framework pretrained Code LLM models code benchmarks language-specific parsers utility functions