title: StructLM: Towards Building Generalist Models for Structured Knowledge
    Grounding
summary: Structured data sources, such as tables, graphs, and databases, are
  ubiquitous knowledge sources. Despite the demonstrated capabilities of large
  language models (LLMs) on plain text, their proficiency in interpreting and
  utilizing structured data remains limited. Our investigation reveals a notable
  deficiency in LLMs' ability to process structured data, e.g., ChatGPT lags
  behind state-of-the-art (SoTA) model by an average of 35%. To augment the
  Structured Knowledge Grounding (SKG) capabilities in LLMs, we have developed a
  comprehensive instruction tuning dataset comprising 1.1 million examples.
  Utilizing this dataset, we train a series of models, referred to as StructLM,
  based on the Code-LLaMA architecture, ranging from 7B to 34B parameters. Our
  StructLM series surpasses task-specific models on 14 out of 18 evaluated
  datasets and establishes new SoTA achievements on 7 SKG tasks. Furthermore,
  StructLM demonstrates exceptional generalization across 6 novel SKG tasks.
  Contrary to expectations, we observe that scaling model size offers marginal
  benefits, with StructLM-34B showing only slight improvements over StructLM-7B.
  This suggests that structured knowledge grounding is still a challenging task
  and requires more innovative design to push to a new level.
paper_authors._id: 65dd73ce3ae2cf0ab2138a19
paper_authors.name: Alex Zhuang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-27T10:35:06.206Z
paper_authors._id: 6498eecc3756a91498f08f60
paper_authors.avatarUrl: /avatars/d92a98535d212c73b9249aa48c717f5e.svg
paper_authors.fullname: Zhuang
paper_authors.type: user
paper_authors.user: AlexZhuang
paper_authors._id: 65dd73ce3ae2cf0ab2138a1a
paper_authors.name: Ge Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-27T08:11:45.125Z
paper_authors._id: 638efcf4c67af472d316d424
paper_authors.avatarUrl: /avatars/97a57859d7d87a3a8f1bb41d32a72bc2.svg
paper_authors.fullname: Ge Zhang
paper_authors.type: user
paper_authors.user: zhangysk
paper_authors._id: 65dd73ce3ae2cf0ab2138a1b
paper_authors.name: Tianyu Zheng
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-27T08:11:37.158Z
paper_authors._id: 64ab99dcb76bfd863eba64c1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64ab99dcb76bfd863eba64c1/UBXwDPx17X-gl-SzBPvrc.jpeg
paper_authors.fullname: TY.Zheng
paper_authors.type: user
paper_authors.user: aaabiao
paper_authors._id: 65dd73ce3ae2cf0ab2138a1c
paper_authors.name: Xinrun Du
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-05-31T06:53:56.569Z
paper_authors._id: 654907a4a1faff97850c4eff
paper_authors.avatarUrl: /avatars/458c90151614bc7f116943b6e67d6b8a.svg
paper_authors.fullname: du
paper_authors.type: user
paper_authors.user: dododododo
paper_authors._id: 65dd73ce3ae2cf0ab2138a1d
paper_authors.name: Junjie Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-28T08:28:13.932Z
paper_authors._id: 62579c55b98dcaa7e0de285d
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/62579c55b98dcaa7e0de285d/0YUd5nloul_bW9yolDGGo.jpeg
paper_authors.fullname: wangjunjie
paper_authors.type: user
paper_authors.user: wanng
paper_authors._id: 65dd73ce3ae2cf0ab2138a1e
paper_authors.name: Weiming Ren
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-27T10:35:43.147Z
paper_authors._id: 64405a9d518271b0d1beea38
paper_authors.avatarUrl: /avatars/b702474588fd7090773320422417a582.svg
paper_authors.fullname: Weiming Ren
paper_authors.type: user
paper_authors.user: wren93
paper_authors._id: 65dd73ce3ae2cf0ab2138a1f
paper_authors.name: Stephen W. Huang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-13T20:26:13.768Z
paper_authors._id: 65bb11cb00a03997849e9e85
paper_authors.avatarUrl: /avatars/17022b0254192a837f4fe00d84389cda.svg
paper_authors.fullname: Wenhao Huang
paper_authors.type: user
paper_authors.user: StephenHuang
paper_authors._id: 65dd73ce3ae2cf0ab2138a20
paper_authors.name: Jie Fu
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-02-27T09:04:26.312Z
paper_authors._id: 641a6895fb5ffff5ac79d593
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/641a6895fb5ffff5ac79d593/dFR_ofjbqCrcqGa9R3MMq.jpeg
paper_authors.fullname: Jie Fu
paper_authors.type: user
paper_authors.user: bigaidream
paper_authors._id: 65dd73ce3ae2cf0ab2138a21
paper_authors.name: Xiang Yue
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-27T10:36:34.221Z
paper_authors._id: 6230d750d93e84e233882dbc
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/6230d750d93e84e233882dbc/4MGEekLW3oWzqeFWDWvIK.jpeg
paper_authors.fullname: Xiang Yue
paper_authors.type: user
paper_authors.user: yuexiang96
paper_authors._id: 65dd73ce3ae2cf0ab2138a22
paper_authors.name: Wenhu Chen
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-02-27T10:36:40.228Z
paper_authors._id: 6313a86154e6e5d9f0f94e04
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1662232951344-6313a86154e6e5d9f0f94e04.jpeg
paper_authors.fullname: Wenhu Chen
paper_authors.type: user
paper_authors.user: wenhu
paper_title: StructLM: Towards Building Generalist Models for Structured Knowledge
    Grounding
paper_summary: Structured data sources, such as tables, graphs, and databases, are
  ubiquitous knowledge sources. Despite the demonstrated capabilities of large
  language models (LLMs) on plain text, their proficiency in interpreting and
  utilizing structured data remains limited. Our investigation reveals a notable
  deficiency in LLMs' ability to process structured data, e.g., ChatGPT lags
  behind state-of-the-art (SoTA) model by an average of 35%. To augment the
  Structured Knowledge Grounding (SKG) capabilities in LLMs, we have developed a
  comprehensive instruction tuning dataset comprising 1.1 million examples.
  Utilizing this dataset, we train a series of models, referred to as StructLM,
  based on the Code-LLaMA architecture, ranging from 7B to 34B parameters. Our
  StructLM series surpasses task-specific models on 14 out of 18 evaluated
  datasets and establishes new SoTA achievements on 7 SKG tasks. Furthermore,
  StructLM demonstrates exceptional generalization across 6 novel SKG tasks.
  Contrary to expectations, we observe that scaling model size offers marginal
  benefits, with StructLM-34B showing only slight improvements over StructLM-7B.
  This suggests that structured knowledge grounding is still a challenging task
  and requires more innovative design to push to a new level.
paper_ai_keywords: large language models (LLMs) structured data tables graphs databases labeled capabilities plain text interpretation utilization structured data processing Structured Knowledge Grounding (SKG) instruction tuning dataset Code-LLaMA architecture Gen4 34B parameters task-specific models SoTA achievements generalization