  Advances in Natural Language Processing (NLP) have revolutionized the way
researchers and practitioners address crucial societal problems. Large language
models are now the standard to develop state-of-the-art solutions for text
detection and classification tasks. However, the development of advanced
computational techniques and resources is disproportionately focused on the
English language, sidelining a majority of the languages spoken globally. While
existing research has developed better multilingual and monolingual language
models to bridge this language disparity between English and non-English
languages, we explore the promise of incorporating the information contained in
images via multimodal machine learning. Our comparative analyses on three
detection tasks focusing on crisis information, fake news, and emotion
recognition, as well as five high-resource non-English languages, demonstrate
that: (a) detection frameworks based on pre-trained large language models like
BERT and multilingual-BERT systematically perform better on the English
language compared against non-English languages, and (b) including images via
multimodal learning bridges this performance gap. We situate our findings with
respect to existing work on the pitfalls of large language models, and discuss
their theoretical and practical implications. Resources for this paper are
available at https://multimodality-language-disparity.github.io/.

Machine Learning;Computers and Society;Multimedia