title: EBES: Easy Benchmarking for Event Sequences
summary: Event sequences, characterized by irregular sampling intervals and a mix of
  categorical and numerical features, are common data structures in various
  real-world domains such as healthcare, finance, and user interaction logs.
  Despite advances in temporal data modeling techniques, there is no standardized
  benchmarks for evaluating their performance on event sequences. This
  complicates result comparison across different papers due to varying evaluation
  protocols, potentially misleading progress in this field. We introduce EBES, a
  comprehensive benchmarking tool with standardized evaluation scenarios and
  protocols, focusing on regression and classification problems with
  sequence-level targets. Our library simplifies benchmarking, dataset addition,
  and method integration through a unified interface. It includes a novel
  synthetic dataset and provides preprocessed real-world datasets, including the
  largest publicly available banking dataset. Our results provide an in-depth
  analysis of datasets, identifying some as unsuitable for model comparison. We
  investigate the importance of modeling temporal and sequential components, as
  well as the robustness and scaling properties of the models. These findings
  highlight potential directions for future research. Our benchmark aim is to
  facilitate reproducible research, expediting progress and increasing real-world
  impacts.
paper_authors._id: 670673a010de8dee36ac7069
paper_authors.name: Dmitry Osin
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-10-21T12:56:07.020Z
paper_authors._id: 67164dea24552afb831fcb10
paper_authors.avatarUrl: /avatars/97be1764f70e27634702f49be717576b.svg
paper_authors.fullname: Dmitry Osin
paper_authors.type: user
paper_authors.user: xaosina
paper_authors._id: 670673a010de8dee36ac706a
paper_authors.name: Igor Udovichenko
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2025-05-26T08:45:49.043Z
paper_authors._id: 67a33f0e36fccbd55d6e8f7f
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/67a33f0e36fccbd55d6e8f7f/o_2UU5bxtnPmPdKo-jg7x.png
paper_authors.fullname: Igor Udovichenko
paper_authors.type: user
paper_authors.user: i-udovichenko
paper_authors._id: 670673a010de8dee36ac706b
paper_authors.name: Viktor Moskvoretskii
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2024-10-09T13:12:12.389Z
paper_authors._id: 63bbfd74141c7d395c471768
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1673264437106-noauth.jpeg
paper_authors.fullname: Viktor Moskvoretskii
paper_authors.type: user
paper_authors.user: VityaVitalich
paper_authors._id: 670673a010de8dee36ac706c
paper_authors.name: Egor Shvetsov
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-10-13T08:00:06.230Z
paper_authors._id: 65afde6ba0b4bf3b0e95b4e8
paper_authors.avatarUrl: /avatars/e9b97040b0a619bf6609465d1678705c.svg
paper_authors.fullname: Egor Shvetsov
paper_authors.type: user
paper_authors.user: dalime
paper_authors._id: 670673a010de8dee36ac706d
paper_authors.name: Evgeny Burnaev
paper_title: EBES: Easy Benchmarking for Event Sequences
paper_summary: Event sequences, characterized by irregular sampling intervals and a mix of
  categorical and numerical features, are common data structures in various
  real-world domains such as healthcare, finance, and user interaction logs.
  Despite advances in temporal data modeling techniques, there is no standardized
  benchmarks for evaluating their performance on event sequences. This
  complicates result comparison across different papers due to varying evaluation
  protocols, potentially misleading progress in this field. We introduce EBES, a
  comprehensive benchmarking tool with standardized evaluation scenarios and
  protocols, focusing on regression and classification problems with
  sequence-level targets. Our library simplifies benchmarking, dataset addition,
  and method integration through a unified interface. It includes a novel
  synthetic dataset and provides preprocessed real-world datasets, including the
  largest publicly available banking dataset. Our results provide an in-depth
  analysis of datasets, identifying some as unsuitable for model comparison. We
  investigate the importance of modeling temporal and sequential components, as
  well as the robustness and scaling properties of the models. These findings
  highlight potential directions for future research. Our benchmark aim is to
  facilitate reproducible research, expediting progress and increasing real-world
  impacts.
paper_ai_summary: EBES, a standardized benchmarking tool, evaluates performance on event sequences across regression and classification tasks, enhancing reproducibility and identifying areas for future research.
paper_ai_keywords: event sequences EBES standardized benchmarks synthetic dataset sequence-level targets reproducible research