OpenVNA: A Framework for Analyzing the Behavior of Multimodal Language Understanding System under Noisy Scenarios
<|reference_start|>OpenVNA: A Framework for Analyzing the Behavior of Multimodal Language Understanding System under Noisy Scenarios: We present OpenVNA, an open-source framework designed for analyzing the behavior of multimodal language understanding systems under noisy conditions. OpenVNA serves as an intuitive toolkit tailored for researchers, facilitating convenience batch-level robustness evaluation and on-the-fly instance-level demonstration. It primarily features a benchmark Python library for assessing global model robustness, offering high flexibility and extensibility, thereby enabling customization with user-defined noise types and models. Additionally, a GUI-based interface has been developed to intuitively analyze local model behavior. In this paper, we delineate the design principles and utilization of the created library and GUI-based web platform. Currently, OpenVNA is publicly accessible at \url{https://github.com/thuiar/OpenVNA}, with a demonstration video available at \url{https://youtu.be/0Z9cW7RGct4}.<|reference_end|>
@article{yuan2024openvna:,
    title={OpenVNA: A Framework for Analyzing the Behavior of Multimodal Language
  Understanding System under Noisy Scenarios},
    author={Ziqi Yuan, Baozheng Zhang, Hua Xu, Zhiyun Liang, and Kai Gao},
    journal={arXiv preprint arXiv:2407.02773},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.02773},
    primaryClass={cs.MM}
}