title: TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language
    Modeling Likewise
summary: Large Language Models (LLMs) exhibit impressive reasoning and data
  augmentation capabilities in various NLP tasks. However, what about small
  models? In this work, we propose TeacherLM-7.1B, capable of annotating relevant
  fundamentals, chain of thought, and common mistakes for most NLP samples, which
  makes annotation more than just an answer, thus allowing other models to learn
  "why" instead of just "what". The TeacherLM-7.1B model achieved a zero-shot
  score of 52.3 on MMLU, surpassing most models with over 100B parameters. Even
  more remarkable is its data augmentation ability. Based on TeacherLM-7.1B, we
  augmented 58 NLP datasets and taught various student models with different
  parameters from OPT and BLOOM series in a multi-task setting. The experimental
  results indicate that the data augmentation provided by TeacherLM has brought
  significant benefits. We will release the TeacherLM series of models and
  augmented datasets as open-source.
paper_authors._id: 65408bfa99e887c0405a6956
paper_authors.name: Nan He
paper_authors._id: 65408bfa99e887c0405a6957
paper_authors.name: Hanyu Lai
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-31T09:42:47.526Z
paper_authors._id: 62ea2d48b493272c269e8f34
paper_authors.avatarUrl: /avatars/89b215dafa503b51ab212a9b63c82aca.svg
paper_authors.fullname: Hanyu Lai
paper_authors.type: user
paper_authors.user: hanyullai
paper_authors._id: 65408bfa99e887c0405a6958
paper_authors.name: Chenyang Zhao
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-31T09:42:58.413Z
paper_authors._id: 64d457059ba46988ac54d40e
paper_authors.avatarUrl: /avatars/4b12a2cda302c38f931dd7555c233daa.svg
paper_authors.fullname: Chenyang Zhao
paper_authors.type: user
paper_authors.user: cyzhao
paper_authors._id: 65408bfa99e887c0405a6959
paper_authors.name: Zirui Cheng
paper_authors._id: 65408bfa99e887c0405a695a
paper_authors.name: Junting Pan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-31T09:43:37.893Z
paper_authors._id: 639c2d36e62c431aa8d834a7
paper_authors.avatarUrl: /avatars/a23d57dcb190711494a2a7e75e4cb9a8.svg
paper_authors.fullname: Junting Pan
paper_authors.type: user
paper_authors.user: juntingpan
paper_authors._id: 65408bfa99e887c0405a695b
paper_authors.name: Ruoyu Qin
paper_authors._id: 65408bfa99e887c0405a695c
paper_authors.name: Ruofan Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-31T09:43:52.843Z
paper_authors._id: 63203f0a7f395c502abcfcde
paper_authors.avatarUrl: /avatars/e6a4df28ce3fc7c6a5cfbbd60bf5563a.svg
paper_authors.fullname: Ruofan Lu
paper_authors.type: user
paper_authors.user: lurf21
paper_authors._id: 65408bfa99e887c0405a695d
paper_authors.name: Rui Lu
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-11-03T09:03:11.381Z
paper_authors._id: 63ca0433b047b9b12a1fcd9c
paper_authors.avatarUrl: /avatars/435507115bfc502c494989677cfca5fb.svg
paper_authors.fullname: Rui Lu
paper_authors.type: user
paper_authors.user: learning-rate
paper_authors._id: 65408bfa99e887c0405a695e
paper_authors.name: Yunchen Zhang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-31T09:44:18.803Z
paper_authors._id: 6204bd1e12ccba127e39aa94
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1663764189519-6204bd1e12ccba127e39aa94.jpeg
paper_authors.fullname: Yunchen Zhang
paper_authors.type: user
paper_authors.user: yunchen
paper_authors._id: 65408bfa99e887c0405a695f
paper_authors.name: Gangming Zhao
paper_authors._id: 65408bfa99e887c0405a6960
paper_authors.name: Zhaohui Hou
paper_authors._id: 65408bfa99e887c0405a6961
paper_authors.name: Zhiyuan Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-31T09:44:59.620Z
paper_authors._id: 64f5a2e46a89a9fc0dc6f470
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64f5a2e46a89a9fc0dc6f470/L0thrI2El7RhXCf6W0gVz.jpeg
paper_authors.fullname: huangzhiyuan
paper_authors.type: user
paper_authors.user: huangzhiyuan
paper_authors._id: 65408bfa99e887c0405a6962
paper_authors.name: Shaoqing Lu
paper_authors._id: 65408bfa99e887c0405a6963
paper_authors.name: Ding Liang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2023-10-31T09:45:13.971Z
paper_authors._id: 6476b01cbb7fdd7f425dcefb
paper_authors.avatarUrl: /avatars/65efe7067f47a68c204a1ab5a772b939.svg
paper_authors.fullname: dingliang
paper_authors.type: user
paper_authors.user: dingliang01
paper_authors._id: 65408bfa99e887c0405a6964
paper_authors.name: Mingjie Zhan
paper_title: TeacherLM: Teaching to Fish Rather Than Giving the Fish, Language
    Modeling Likewise
paper_summary: Large Language Models (LLMs) exhibit impressive reasoning and data
  augmentation capabilities in various NLP tasks. However, what about small
  models? In this work, we propose TeacherLM-7.1B, capable of annotating relevant
  fundamentals, chain of thought, and common mistakes for most NLP samples, which
  makes annotation more than just an answer, thus allowing other models to learn
  "why" instead of just "what". The TeacherLM-7.1B model achieved a zero-shot
  score of 52.3 on MMLU, surpassing most models with over 100B parameters. Even
  more remarkable is its data augmentation ability. Based on TeacherLM-7.1B, we
  augmented 58 NLP datasets and taught various student models with different
  parameters from OPT and BLOOM series in a multi-task setting. The experimental
  results indicate that the data augmentation provided by TeacherLM has brought
  significant benefits. We will release the TeacherLM series of models and
  augmented datasets as open-source.
paper_ai_keywords: TeacherLM-7.1B chain of thought zero-shot MMLU data augmentation OPT BLOOM series multi-task setting