  Geometric deep learning (GDL) has gained significant attention in various
scientific fields, chiefly for its proficiency in modeling data with intricate
geometric structures. Yet, very few works have delved into its capability of
tackling the distribution shift problem, a prevalent challenge in many relevant
applications. To bridge this gap, we propose GDL-DS, a comprehensive benchmark
designed for evaluating the performance of GDL models in scenarios with
distribution shifts. Our evaluation datasets cover diverse scientific domains
from particle physics and materials science to biochemistry, and encapsulate a
broad spectrum of distribution shifts including conditional, covariate, and
concept shifts. Furthermore, we study three levels of information access from
the out-of-distribution (OOD) testing data, including no OOD information, only
OOD features without labels, and OOD features with a few labels. Overall, our
benchmark results in 30 different experiment settings, and evaluates 3 GDL
backbones and 11 learning algorithms in each setting. A thorough analysis of
the evaluation results is provided, poised to illuminate insights for DGL
researchers and domain practitioners who are to use DGL in their applications.

Machine Learning;Artificial Intelligence