GenVP: Generating Visual Puzzles with Contrastive Hierarchical VAEs
Kalliopi Basioti Pritish Sahu Tony Qingze Liu Zihao Xu Hao Wang Vladimir Pavlovic
Raven’s Progressive Matrices (RPMs) is an established benchmark to examine
the ability to perform high-level abstract visual reasoning (AVR). Despite the current success of algorithms that solve this task, humans can generalize beyond a given puzzle and create new puzzles given a set of rules, whereas machines remain locked in solving a fixed puzzle from a curated choice list. We propose Generative Visual Puzzles (GenVP), a framework to model the entire RPM generation process, a substantially more challenging task. Our model’s capability spans from generating multiple solutions for one specific problem prompt to creating complete new puzzles out of the desired set of rules. Experiments on five different datasets indicate that GenVP achieves state-of-the-art (SOTA) performance both in puzzle-solving accuracy and out-of-distribution (OOD) generalization in 22 out
of 24 OOD scenarios. Further, compared to SOTA generative approaches, which struggle to solve RPMs when the feasible solution space increases, GenVP efficiently generalizes to these challenging scenarios. Moreover, our model demonstrates the ability to produce a wide range of complete RPMs given a set of abstract rules by effectively capturing the relationships between abstract rules and visual object properties.
abstract visual reasoning; contrastive learning; variational autoencoders; out of distribution generalization
We propose Generative Visual Puzzles (GenVP), a generative RPM solver exceeding SOTA accuracy and capable of generate new RPM matrix instances different from original ones while following the same rules.