  Concept bottleneck models (CBMs) are a class of interpretable neural network
models that predict the target response of a given input based on its
high-level concepts. Unlike the standard end-to-end models, CBMs enable domain
experts to intervene on the predicted concepts and rectify any mistakes at test
time, so that more accurate task predictions can be made at the end. While such
intervenability provides a powerful avenue of control, many aspects of the
intervention procedure remain rather unexplored. In this work, we develop
various ways of selecting intervening concepts to improve the intervention
effectiveness and conduct an array of in-depth analyses as to how they evolve
under different circumstances. Specifically, we find that an informed
intervention strategy can reduce the task error more than ten times compared to
the current baseline under the same amount of intervention counts in realistic
settings, and yet, this can vary quite significantly when taking into account
different intervention granularity. We verify our findings through
comprehensive evaluations, not only on the standard real datasets, but also on
synthetic datasets that we generate based on a set of different causal graphs.
We further discover some major pitfalls of the current practices which, without
a proper addressing, raise concerns on reliability and fairness of the
intervention procedure.

Machine Learning