Two important topics in deep learning both involve incorporating humans into the modeling process: Model priors transfer information from humans to a model by regularizing the model's parameters; Model attributions transfer information from a model to humans by explaining the model's behavior. Previous work has taken important steps to connect these topics through gradient regularization, but we find that these methods do not successfully align a model's behavior with human intuition. We develop an efficient and theoretically grounded feature attribution method, expected gradients, and a novel framework, attribution priors, to enforce prior expectations about a model's behavior during training. We demonstrate that attribution priors are broadly applicable in three different domains: image data, gene expression data, and health care data. Our experiments show that models trained with attribution priors are more intuitive and achieve better generalization performance than both equivalent baselines and existing methods to regularize model behavior.
Summary.
The paper improves the existing feature attribution method by adding regularizers to enforce (human) expectations about a model’s behavior. Three different datasets (i.e. image, gene expression, health-care) are chosen to evaluate the proposed model’s effectiveness, while different regularizers (i.e. image prior, graph prior, and sparsity prior) are explored for the respective task. 
Strengths.
1. Incorporating human knowledge into the model has a growing interest in ML / CV communities.
2. Three datasets from different domains (i.e. image classification data, gene expression data, and health care data) are used to evaluate the effectiveness of the proposed approach. Data shows that the proposed approach shows better generalization performance (i.e. better performance in test dataset) than baselines.
3. The paper provides well-documented supplemental materials that contain details of the experimental setting and additional supporting figures.
Weaknesses.
1. Task-specific heuristic human prior
I agree (and personally like) the motivation that a method is needed to align a model’s behavior with human knowledge or intuition -- model’s behavior may be explained by feature attribution methods while making models accept human knowledge is challenging. However, such an ability is achieved by simply adding task-specific heuristic functions as a penalty or a regularizer. Also, the introduced human priors are similar to general regularization conventions, i.e. a penalty of smoothness over adjacent pixels is commonly used in the CV community. I am concerned that only a limited set of expert-invented human priors can be used in this approach.
Further, feature attribution methods aim to develop a richer notion of the contribution of a pixel to the output. However, the difficulty would be the lack of formal measures of how the network output is affected by spatially-extended features (rather than pixels). The explored priors (e.g. a total variation loss to make neighboring pixels have a similar impact on the final verdict) actually relieve this issue.
2. Incorporating humans into the modeling process?
A key motivation behind this work is “incorporating humans into the modeling process”. This would imply that (human-understandable) information needs first to be transferred from a model to humans. However, I am concerned about what information end-users are expected to obtain from the model. For example, Figure 1 (left) shows an attribution map that highlights multiple intermittent regions from which I cannot understand its behavior. Unless end-users cannot understand the model’s behavior, how can we expect humans can provide knowledge to model? A user study would be needed to support that the proposed method can really provide a way to incorporate humans into the modeling process.
Minor comments.
1. Plots in Figure 3 are not intuitively understandable.
2. There is no section Conclusion.
3. A template for the reference section looks different from other ICLR papers.