  The rule extraction literature contains the notion of a fidelity-accuracy
dilemma: when building an interpretable model of a black box function,
optimising for fidelity is likely to reduce performance on the underlying task,
and vice versa. I reassert the relevance of this dilemma for the modern field
of explainable artificial intelligence, and highlight how it is compounded when
the black box is an agent interacting with a dynamic environment. I then
discuss two independent research directions - building white box agents and
interpreting black box agents - which are both coherent and worthy of
attention, but must not be conflated by researchers embarking on projects in
the domain of agent interpretability.

Artificial Intelligence