title: RelBench: A Benchmark for Deep Learning on Relational Databases
summary: We present RelBench, a public benchmark for solving predictive tasks over
  relational databases with graph neural networks. RelBench provides databases
  and tasks spanning diverse domains and scales, and is intended to be a
  foundational infrastructure for future research. We use RelBench to conduct the
  first comprehensive study of Relational Deep Learning (RDL) (Fey et al., 2024),
  which combines graph neural network predictive models with (deep) tabular
  models that extract initial entity-level representations from raw tables.
  End-to-end learned RDL models fully exploit the predictive signal encoded in
  primary-foreign key links, marking a significant shift away from the dominant
  paradigm of manual feature engineering combined with tabular models. To
  thoroughly evaluate RDL against this prior gold-standard, we conduct an
  in-depth user study where an experienced data scientist manually engineers
  features for each task. In this study, RDL learns better models whilst reducing
  human work needed by more than an order of magnitude. This demonstrates the
  power of deep learning for solving predictive tasks over relational databases,
  opening up many new research opportunities enabled by RelBench.
paper_authors._id: 66a9e7ed85f81589aaee8291
paper_authors.name: Joshua Robinson
paper_authors._id: 66a9e7ed85f81589aaee8292
paper_authors.name: Rishabh Ranjan
paper_authors._id: 66a9e7ed85f81589aaee8293
paper_authors.name: Weihua Hu
paper_authors._id: 66a9e7ed85f81589aaee8294
paper_authors.name: Kexin Huang
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-05T09:35:40.752Z
paper_authors._id: 631fc126788f34547a9d7ee1
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1674202999503-631fc126788f34547a9d7ee1.png
paper_authors.fullname: Kexin Huang
paper_authors.type: user
paper_authors.user: kexinhuang
paper_authors._id: 66a9e7ed85f81589aaee8295
paper_authors.name: Jiaqi Han
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-21T08:11:12.695Z
paper_authors._id: 66ce1661d620749ad81dda84
paper_authors.avatarUrl: /avatars/e8a454c3d45b2e878388b1a6e466a5a1.svg
paper_authors.fullname: Jiaqi Han
paper_authors.type: user
paper_authors.user: jiaqihan99
paper_authors._id: 66a9e7ed85f81589aaee8296
paper_authors.name: Alejandro Dobles
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-05T09:35:15.329Z
paper_authors._id: 65d23685f94e504965d92721
paper_authors.avatarUrl: /avatars/4c76c11ed975e0e432e7f6ab24ee99a8.svg
paper_authors.fullname: Alejandro Dobles
paper_authors.type: user
paper_authors.user: adobles
paper_authors._id: 66a9e7ed85f81589aaee8297
paper_authors.name: Matthias Fey
paper_authors._id: 66a9e7ed85f81589aaee8298
paper_authors.name: Jan E. Lenssen
paper_authors._id: 66a9e7ed85f81589aaee8299
paper_authors.name: Yiwen Yuan
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-08-05T09:34:58.016Z
paper_authors._id: 64e43a6382fb6ae7a08afdbb
paper_authors.avatarUrl: /avatars/4876d26cfb4acbb31ad52b971b67e77a.svg
paper_authors.fullname: Yiwen Yuan
paper_authors.type: user
paper_authors.user: yiweny
paper_authors._id: 66a9e7ed85f81589aaee829a
paper_authors.name: Zecheng Zhang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-31T07:43:30.330Z
paper_authors._id: 609a5543456ee4a3392e8427
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1620727835835-609a5543456ee4a3392e8427.jpeg
paper_authors.fullname: Zecheng Zhang
paper_authors.type: user
paper_authors.user: zechengz
paper_authors._id: 66a9e7ed85f81589aaee829b
paper_authors.name: Xinwei He
paper_authors._id: 66a9e7ed85f81589aaee829c
paper_authors.name: Jure Leskovec
paper_title: RelBench: A Benchmark for Deep Learning on Relational Databases
paper_summary: We present RelBench, a public benchmark for solving predictive tasks over
  relational databases with graph neural networks. RelBench provides databases
  and tasks spanning diverse domains and scales, and is intended to be a
  foundational infrastructure for future research. We use RelBench to conduct the
  first comprehensive study of Relational Deep Learning (RDL) (Fey et al., 2024),
  which combines graph neural network predictive models with (deep) tabular
  models that extract initial entity-level representations from raw tables.
  End-to-end learned RDL models fully exploit the predictive signal encoded in
  primary-foreign key links, marking a significant shift away from the dominant
  paradigm of manual feature engineering combined with tabular models. To
  thoroughly evaluate RDL against this prior gold-standard, we conduct an
  in-depth user study where an experienced data scientist manually engineers
  features for each task. In this study, RDL learns better models whilst reducing
  human work needed by more than an order of magnitude. This demonstrates the
  power of deep learning for solving predictive tasks over relational databases,
  opening up many new research opportunities enabled by RelBench.
paper_ai_keywords: RelBench Relational Deep Learning (RDL) graph neural networks tabular models primary-foreign key links manual feature engineering GroundScore