EBES: Easy Benchmarking for Event Sequences
Event sequences, characterized by irregular sampling intervals and a mix of
categorical and numerical features, are common data structures in various
real-world domains such as healthcare, finance, and user interaction logs.
Despite advances in temporal data modeling techniques, there is no standardized
benchmarks for evaluating their performance on event sequences. This
complicates result comparison across different papers due to varying evaluation
protocols, potentially misleading progress in this field. We introduce EBES, a
comprehensive benchmarking tool with standardized evaluation scenarios and
protocols, focusing on regression and classification problems with
sequence-level targets. Our library simplifies benchmarking, dataset addition,
and method integration through a unified interface. It includes a novel
synthetic dataset and provides preprocessed real-world datasets, including the
largest publicly available banking dataset. Our results provide an in-depth
analysis of datasets, identifying some as unsuitable for model comparison. We
investigate the importance of modeling temporal and sequential components, as
well as the robustness and scaling properties of the models. These findings
highlight potential directions for future research. Our benchmark aim is to
facilitate reproducible research, expediting progress and increasing real-world
impacts.
670673a010de8dee36ac7069
Dmitry Osin
claimed_verified
2024-10-21T12:56:07.020Z
67164dea24552afb831fcb10
/avatars/97be1764f70e27634702f49be717576b.svg
Dmitry Osin
user
xaosina
670673a010de8dee36ac706a
Igor Udovichenko
claimed_verified
2025-05-26T08:45:49.043Z
67a33f0e36fccbd55d6e8f7f
https://cdn-avatars.huggingface.co/v1/production/uploads/67a33f0e36fccbd55d6e8f7f/o_2UU5bxtnPmPdKo-jg7x.png
Igor Udovichenko
user
i-udovichenko
670673a010de8dee36ac706b
Viktor Moskvoretskii
admin_assigned
2024-10-09T13:12:12.389Z
63bbfd74141c7d395c471768
https://cdn-avatars.huggingface.co/v1/production/uploads/1673264437106-noauth.jpeg
Viktor Moskvoretskii
user
VityaVitalich
670673a010de8dee36ac706c
Egor Shvetsov
extracted_confirmed
2024-10-13T08:00:06.230Z
65afde6ba0b4bf3b0e95b4e8
/avatars/e9b97040b0a619bf6609465d1678705c.svg
Egor Shvetsov
user
dalime
670673a010de8dee36ac706d
Evgeny Burnaev
EBES: Easy Benchmarking for Event Sequences
Event sequences, characterized by irregular sampling intervals and a mix of
categorical and numerical features, are common data structures in various
real-world domains such as healthcare, finance, and user interaction logs.
Despite advances in temporal data modeling techniques, there is no standardized
benchmarks for evaluating their performance on event sequences. This
complicates result comparison across different papers due to varying evaluation
protocols, potentially misleading progress in this field. We introduce EBES, a
comprehensive benchmarking tool with standardized evaluation scenarios and
protocols, focusing on regression and classification problems with
sequence-level targets. Our library simplifies benchmarking, dataset addition,
and method integration through a unified interface. It includes a novel
synthetic dataset and provides preprocessed real-world datasets, including the
largest publicly available banking dataset. Our results provide an in-depth
analysis of datasets, identifying some as unsuitable for model comparison. We
investigate the importance of modeling temporal and sequential components, as
well as the robustness and scaling properties of the models. These findings
highlight potential directions for future research. Our benchmark aim is to
facilitate reproducible research, expediting progress and increasing real-world
impacts.
EBES, a standardized benchmarking tool, evaluates performance on event sequences across regression and classification tasks, enhancing reproducibility and identifying areas for future research.
event sequences EBES standardized benchmarks synthetic dataset sequence-level targets reproducible research