  Active learning aims to construct an effective training set by iteratively
curating the most informative unlabeled data for annotation, which is practical
in low-resource tasks. Most active learning techniques in classification rely
on the model's uncertainty or disagreement to choose unlabeled data. However,
previous work indicates that existing models are poor at quantifying predictive
uncertainty, which can lead to over-confidence in superficial patterns and a
lack of exploration. Inspired by the cognitive processes in which humans deduce
and predict through causal information, we propose a novel Explainable Active
Learning framework (XAL) for low-resource text classification, which aims to
encourage classifiers to justify their inferences and delve into unlabeled data
for which they cannot provide reasonable explanations. Specifically, besides
using a pre-trained bi-directional encoder for classification, we employ a
pre-trained uni-directional decoder to generate and score the explanation. A
ranking loss is proposed to enhance the decoder's capability in scoring
explanations. During the selection of unlabeled data, we combine the predictive
uncertainty of the encoder and the explanation score of the decoder to acquire
informative data for annotation.
  As XAL is a general framework for text classification, we test our methods on
six different classification tasks. Extensive experiments show that XAL
achieves substantial improvement on all six tasks over previous AL methods.
Ablation studies demonstrate the effectiveness of each component, and human
evaluation shows that the model trained in XAL performs surprisingly well in
explaining its prediction.

Computation and Language