ViPro: Enabling and Controlling Video Prediction for Complex Dynamical Scenarios using Procedural Knowledge
<|reference_start|>ViPro: Enabling and Controlling Video Prediction for Complex Dynamical Scenarios using Procedural Knowledge: We propose a novel architecture design for video prediction in order to utilize procedural domain knowledge directly as part of the computational graph of data-driven models. On the basis of new challenging scenarios we show that state-of-the-art video predictors struggle in complex dynamical settings, and highlight that the introduction of prior process knowledge makes their learning problem feasible. Our approach results in the learning of a symbolically addressable interface between data-driven aspects in the model and our dedicated procedural knowledge module, which we utilize in downstream control tasks.<|reference_end|>
@article{takenaka2024vipro:,
    title={ViPro: Enabling and Controlling Video Prediction for Complex Dynamical
  Scenarios using Procedural Knowledge},
    author={Patrick Takenaka, Johannes Maucher, Marco F. Huber},
    journal={arXiv preprint arXiv:2407.09537},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.09537},
    primaryClass={cs.CV cs.AI cs.LG}
}