Tabby: Tabular Data Synthesis with Language Models
While advances in large language models (LLMs) have greatly improved the
quality of synthetic text data in recent years, synthesizing tabular data has
received relatively less attention. We address this disparity with Tabby, a
simple but powerful post-training modification to the standard Transformer
language model architecture, enabling its use for tabular dataset synthesis.
Tabby enables the representation of differences across columns using Gated
Mixture-of-Experts, with column-specific sets of parameters. Empirically, Tabby
results in data quality near or equal to that of real data. By pairing our
novel LLM table training technique, Plain, with Tabby, we observe up to a 44%
improvement in quality over previous methods. We also show that Tabby extends
beyond tables to more general structured data, reaching parity with real data
on a nested JSON dataset as well.
67c8691bb43e75ced9747fe6
Sonia Cromp
extracted_confirmed
2025-03-05T15:57:21.571Z
63865c918b5acae8d24ffff3
/avatars/7916f38ce38266a0a3b39a54f75636ff.svg
Sonia
user
sonicc
67c8691bb43e75ced9747fe7
Satya Sai Srinath Namburi GNVV
claimed_verified
2025-06-23T08:33:53.604Z
632be8c900e5ff9ee1277863
/avatars/04170ba241b61c4d50a7622f207c9ddc.svg
Namburi GNVV Satya Sai Srinath
user
NamburiSrinath
67c8691bb43e75ced9747fe8
Mohammed Alkhudhayri
67c8691bb43e75ced9747fe9
Catherine Cao
67c8691bb43e75ced9747fea
Samuel Guo
67c8691bb43e75ced9747feb
Nicholas Roberts
67c8691bb43e75ced9747fec
Frederic Sala
Tabby: Tabular Data Synthesis with Language Models
While advances in large language models (LLMs) have greatly improved the
quality of synthetic text data in recent years, synthesizing tabular data has
received relatively less attention. We address this disparity with Tabby, a
simple but powerful post-training modification to the standard Transformer
language model architecture, enabling its use for tabular dataset synthesis.
Tabby enables the representation of differences across columns using Gated
Mixture-of-Experts, with column-specific sets of parameters. Empirically, Tabby
results in data quality near or equal to that of real data. By pairing our
novel LLM table training technique, Plain, with Tabby, we observe up to a 44%
improvement in quality over previous methods. We also show that Tabby extends
beyond tables to more general structured data, reaching parity with real data
on a nested JSON dataset as well.
Tabby, a post-training modification using Gated Mixture-of-Experts, enhances Transformer models for tabular and structured data synthesis, achieving high data quality.
Transformer Gated Mixture-of-Experts Tabby Plain