  There is increasing interest in assessing the linguistic knowledge encoded in
neural representations. A popular approach is to attach a diagnostic classifier
-- or "probe" -- to perform supervised classification from internal
representations. However, how to select a good probe is in debate. Hewitt and
Liang (2019) showed that a high performance on diagnostic classification itself
is insufficient, because it can be attributed to either "the representation
being rich in knowledge", or "the probe learning the task", which Pimentel et
al. (2020) challenged. We show this dichotomy is valid
information-theoretically. In addition, we find that the methods to construct
and select good probes proposed by the two papers, *control task* (Hewitt and
Liang, 2019) and *control function* (Pimentel et al., 2020), are equivalent --
the errors of their approaches are identical (modulo irrelevant terms).
Empirically, these two selection criteria lead to results that highly agree
with each other.

Computation and Language