Learning to Infer Generative Template Programs for Visual Concepts
<|reference_start|>Learning to Infer Generative Template Programs for Visual Concepts: People grasp flexible visual concepts from a few examples. We explore a neurosymbolic system that learns how to infer programs that capture visual concepts in a domain-general fashion. We introduce Template Programs: programmatic expressions from a domain-specific language that specify structural and parametric patterns common to an input concept. Our framework supports multiple concept-related tasks, including few-shot generation and co-segmentation through parsing. We develop a learning paradigm that allows us to train networks that infer Template Programs directly from visual datasets that contain concept groupings. We run experiments across multiple visual domains: 2D layouts, Omniglot characters, and 3D shapes. We find that our method outperforms task-specific alternatives, and performs competitively against domain-specific approaches for the limited domains where they exist.<|reference_end|>
@article{jones2024learning,
    title={Learning to Infer Generative Template Programs for Visual Concepts},
    author={R. Kenny Jones, Siddhartha Chaudhuri, Daniel Ritchie},
    journal={arXiv preprint arXiv:2403.15476},
    year={2024},
    archivePrefix={arXiv},
    eprint={2403.15476},
    primaryClass={cs.CV cs.AI cs.GR cs.LG}
}