  Recent work in computer vision and cognitive reasoning has given rise to an
increasing adoption of the Violation-of-Expectation (VoE) paradigm in synthetic
datasets. Inspired by infant psychology, researchers are now evaluating a
model's ability to label scenes as either expected or surprising with knowledge
of only expected scenes. However, existing VoE-based 3D datasets in physical
reasoning provide mainly vision data with little to no heuristics or inductive
biases. Cognitive models of physical reasoning reveal infants create high-level
abstract representations of objects and interactions. Capitalizing on this
knowledge, we established a benchmark to study physical reasoning by curating a
novel large-scale synthetic 3D VoE dataset armed with ground-truth heuristic
labels of causally relevant features and rules. To validate our dataset in five
event categories of physical reasoning, we benchmarked and analyzed human
performance. We also proposed the Object File Physical Reasoning Network
(OFPR-Net) which exploits the dataset's novel heuristics to outperform our
baseline and ablation models. The OFPR-Net is also flexible in learning an
alternate physical reality, showcasing its ability to learn universal causal
relationships in physical reasoning to create systems with better
interpretability.

Computer Vision and Pattern Recognition;Artificial Intelligence