Guiding Video Prediction with Explicit Procedural Knowledge
<|reference_start|>Guiding Video Prediction with Explicit Procedural Knowledge: We propose a general way to integrate procedural knowledge of a domain into deep learning models. We apply it to the case of video prediction, building on top of object-centric deep models and show that this leads to a better performance than using data-driven models alone. We develop an architecture that facilitates latent space disentanglement in order to use the integrated procedural knowledge, and establish a setup that allows the model to learn the procedural interface in the latent space using the downstream task of video prediction. We contrast the performance to a state-of-the-art data-driven approach and show that problems where purely data-driven approaches struggle can be handled by using knowledge about the domain, providing an alternative to simply collecting more data.<|reference_end|>
@article{takenaka2024guiding,
    title={Guiding Video Prediction with Explicit Procedural Knowledge},
    author={Patrick Takenaka, Johannes Maucher, Marco F. Huber},
    journal={2023 IEEE/CVF International Conference on Computer Vision
  Workshops (ICCVW), Paris, France, 2023, pp. 1076-1084},
    year={2024},
    doi={10.1109/ICCVW60793.2023.00116},
    archivePrefix={arXiv},
    eprint={2406.18220},
    primaryClass={cs.CV cs.AI cs.LG}
}