  Despite recent monumental advances in the field, many Natural Language
Processing (NLP) models still struggle to perform adequately on noisy domains.
We propose a novel probabilistic embedding-level method to improve the
robustness of NLP models. Our method, Robust Embeddings via Distributions
(RED), incorporates information from both noisy tokens and surrounding context
to obtain distributions over embedding vectors that can express uncertainty in
semantic space more fully than any deterministic method. We evaluate our method
on a number of downstream tasks using existing state-of-the-art models in the
presence of both natural and synthetic noise, and demonstrate a clear
improvement over other embedding approaches to robustness from the literature.

Computation and Language;Machine Learning