  Artificial Intelligence (AI) systems are attracting increasing interest in
the medical domain due to their ability to learn complicated tasks that require
human intelligence and expert knowledge. AI systems that utilize
high-performance Natural Language Processing (NLP) models have achieved
state-of-the-art results on a wide variety of clinical text processing
benchmarks. They have even outperformed human accuracy on some tasks. However,
performance evaluation of such AI systems have been limited to accuracy
measures on curated and clean benchmark datasets that may not properly reflect
how robustly these systems can operate in real-world situations. In order to
address this challenge, we introduce and implement a wide variety of
perturbation methods that simulate different types of noise and variability in
clinical text data. While noisy samples produced by these perturbation methods
can often be understood by humans, they may cause AI systems to make erroneous
decisions. Conducting extensive experiments on several clinical text processing
tasks, we evaluated the robustness of high-performance NLP models against
various types of character-level and word-level noise. The results revealed
that the NLP models performance degrades when the input contains small amounts
of noise. This study is a significant step towards exposing vulnerabilities of
AI models utilized in clinical text processing systems. The proposed
perturbation methods can be used in performance evaluation tests to assess how
robustly clinical NLP models can operate on noisy data, in real-world settings.

Computation and Language;Artificial Intelligence