Munir Abukhder, Elliot Ismail, Thomas Dobbs
Improving Burns and Plastic Surgery Induction Programmes: A Departmental Quality Improvement Project
Cureus
development multimedia systems wireless environments rising need artificial intelligence design system properly communicate humans comprehensive understanding various types information humanlike manner therefore paper addresses audiovisual sceneaware dialog system communicate users audiovisual scenes essential understand visual textual information also audio information comprehensive way despite substantial progress multimodal representation learning language visual modalities still two caveats ineffective use auditory information lack interpretability deep learning systems reasoning address issues propose novel audiovisual sceneaware dialog system utilizes set explicit information modality form natural language fused language model natural way leverages transformerbased decoder generate coherent correct response based multimodal knowledge multitask learning setting addition also address way interpreting model responsedriven temporal moment localization method verify system generates response system provides user evidence referred system response process form timestamp scene show superiority proposed model quantitative qualitative measurements compared baseline particular proposed model achieved robust performance even environments using three modalities including audio also conducted extensive experiments investigate proposed model addition obtained stateoftheart performance system response reasoning task