Can Programming Languages Boost Each Other via Instruction Tuning?
When human programmers have mastered a programming language, it would be
easier when they learn a new programming language. In this report, we focus on
exploring whether programming languages can boost each other during the
instruction fine-tuning phase of code large language models. We conduct
extensive experiments of 8 popular programming languages (Python, JavaScript,
TypeScript, C, C++, Java, Go, HTML) on StarCoder. Results demonstrate that
programming languages can significantly improve each other. For example,
CodeM-Python 15B trained on Python is able to increase Java by an absolute
17.95% pass@1 on HumanEval-X. More surprisingly, we found that CodeM-HTML 7B
trained on the HTML corpus can improve Java by an absolute 15.24% pass@1. Our
training data is released at https://github.com/NL2Code/CodeM.
64f145f753c3b575c28114d5
Daoguang Zan
admin_assigned
2023-09-01T12:27:23.411Z
61527edf8b55dbdae72874fa
https://cdn-avatars.huggingface.co/v1/production/uploads/61527edf8b55dbdae72874fa/ZGWSBf_KSrDof6WyMoDMU.jpeg
Daoguang Zan
user
Daoguang
64f145f753c3b575c28114d6
Ailun Yu
admin_assigned
2023-09-01T12:27:33.147Z
61b997ae22e5b0fdd501a0ec
https://cdn-avatars.huggingface.co/v1/production/uploads/1639552935869-noauth.png
Ailun Yu
user
Tannpopo
64f145f753c3b575c28114d7
Bo Shen
64f145f753c3b575c28114d8
Jiaxin Zhang
admin_assigned
2023-09-01T12:28:48.246Z
62610bf040e04009e810357c
/avatars/16056719d57d4fcf7c3d21135e65f1a2.svg
Jiaxin Zhang
user
BerryBiscuits
64f145f753c3b575c28114d9
Taihong Chen
64f145f753c3b575c28114da
Bing Geng
admin_assigned
2023-09-01T12:29:31.663Z
646ec43b8d316fde87b3871a
/avatars/195cc396e160bb65ef051a2e5b6a0ebc.svg
Geng Bing
user
edwingeng
64f145f753c3b575c28114db
Bei Chen
claimed_verified
2024-01-23T13:49:32.202Z
64cc65d786d8dc0caa6ab3cd
/avatars/23dc126f7332825e713807fb1c000ab6.svg
Bei Chen
user
BellaBei
64f145f753c3b575c28114dc
Jichuan Ji
64f145f753c3b575c28114dd
Yafen Yao
64f145f753c3b575c28114de
Yongji Wang
admin_assigned
2023-09-01T12:30:41.849Z
645c3d8a1d2ba8a1cf48683a
/avatars/152eb17d5918edc051804b4cbc508c38.svg
Wang
user
Yongji
64f145f753c3b575c28114df
Qianxiang Wang
admin_assigned
2023-09-01T12:30:51.892Z
632db5eba0fd0d71dcc17290
/avatars/6c7b096d495d1128e19d4fb18789e129.svg
Qianxiang Wang
user
WangQ
Can Programming Languages Boost Each Other via Instruction Tuning?
When human programmers have mastered a programming language, it would be
easier when they learn a new programming language. In this report, we focus on
exploring whether programming languages can boost each other during the
instruction fine-tuning phase of code large language models. We conduct
extensive experiments of 8 popular programming languages (Python, JavaScript,
TypeScript, C, C++, Java, Go, HTML) on StarCoder. Results demonstrate that
programming languages can significantly improve each other. For example,
CodeM-Python 15B trained on Python is able to increase Java by an absolute
17.95% pass@1 on HumanEval-X. More surprisingly, we found that CodeM-HTML 7B
trained on the HTML corpus can improve Java by an absolute 15.24% pass@1. Our
training data is released at https://github.com/NL2Code/CodeM.
Programming languages enhance each other during the instruction fine-tuning phase of code large language models, as demonstrated by significant improvements in test performance across different languages.
instruction fine-tuning code large language models HumanEval-X