Many machine learning algorithms are trained and evaluated by splitting data from a single source into training and test sets. While such focus on in-distribution learning scenarios has led interesting advances, it has not been able to tell if models are relying on dataset biases as shortcuts for successful prediction (e.g., using snow cues for recognising snowmobiles). Such biased models fail to generalise when the bias shifts to a different class. The cross-bias generalisation problem has been addressed by de-biasing training data through augmentation or re-sampling, which are often prohibitive due to the data collection cost (e.g., collecting images of a snowmobile on a desert) and the difficulty of quantifying or expressing biases in the first place. In this work, we propose a novel framework to train a de-biased representation by encouraging it to be different from a set of representations that are biased by design. This tactic is feasible in many scenarios where it is much easier to define a set of biased representations than to define and quantify bias. Our experiments and analyses show that our method discourages models from taking bias shortcuts, resulting in improved performances on de-biased test data.
###  Summary 
The paper proposes a method for regularizing neural networks to mitigate certain known biases from the representations learned by CNNs. The authors look at the setting in which the distribution of biases in the train and test set remains the same, but the distribution of targets given biases changes.
To remove the bias, they propose hand-designing a set of models that rely exclusively on these biases to make predictions. They then train a new unbiased model by making sure that this new model uses sufficiently different information than the biased models to make predictions (By adding a regularization term)
### Decision and reasons
I vote for a weak accept. 
Strengths: 
1: The paper is well written with a clearly defined problem-setting. The proposed method is sound and interesting, and the empirical results, thorough. 
Weaknesses:
2: The solution just pushes the problem of 'learning a model that only uses the true signal to make predictions' to 'learning a set of models that only use the noise to make predictions.' It's not clear why the latter is easier than the former. 
3: The paper does not have any baselines that directly try to remove the bias (Instead of using the two-step process). As a result, it's hard to judge how meaningful the improvements are. 
### Supporting arguments for the reasons for the decision.
Strengths: 
1: The paper does a really good job of defining the problem-setting. Contrasting cross-bias with cross-domain and in-distribution makes the goal of the paper very clear. The notation used to formalize the problem setting in Section 2.1 is also clear and concise. Moreover, the experiments on the toy dataset help clarify the proposed solution whereas experiments on Biased MNIST and Imagenet show that it successfully mitigates the bias. Finally, the authors show the importance of each component of the proposed solution by factor analysis.
Weaknesses:
2: In the most general case, it is not obvious why it is easier to define and learn a set of models that only use noise to make predictions (which is the required first step for their proposed solution) as opposed to learning a model that only uses signal (which is the goal of the problem). These two problems seem equally hard. The paper builds on the premise that in some cases former is easier (i.e. in some cases, it is easier to learn a set of models that use only noise as opposed to learning a debiased model directly). The authors give two such examples (They only explore the first experimentally.) 
1. Learning a model that relies on local texture. They achieve this by limiting the receptive field of the features. 
2. Learning a model that relies on static images to make predictions about actions in videos. 
I feel that the two given examples are very narrow. It would be nice if the authors could identify a broader class of problems for which G is given or can easily be defined. Moreover, even in these two examples, I'm not convinced that the proposed biased models only use B for making predictions. For example, for some classification tasks, the local texture could be part of the signal and not just the bias. Similarly, static images from a video do contain important information for making the prediction. 
3: The authors only compare their method to a baseline that does nothing to debias the representations. Even though this is an important comparison (as it shows that the proposed method can debias the representations), it does not tell the reader how effective the proposed method is compared to other possible solutions. The results would be more meaningful if the authors could include at-least a simple baseline that tries to remove the bias in other ways (For example, they could use the style-transfer baseline used by Geirhos et al., 2019). 
I vote for accepting the paper as a poster. It introduces an interesting approach for debiasing representations. However, due to its narrow scope and missing baselines, I would not recommend the paper for an oral presentation. 
### Questions
1. What are some broader class of problems for which defining G is easier than directly regularizing for debiased representations?
2. How well do Bagnets alone perform on the benchmarks in Table 3? I would expect to see that Bagnets alone do worse than vanilla Resnets on Unbiased and IN-A. Is that so? 
3. How well do other methods do in these domains? (Such as methods that directly debias the training data against texture by applying style-transfer). 
### Update after Author's response
The author's response has clarified the motivation behind the proposed approach to an extent. They have also added a comparison with a method that directly promotes learning the shape as opposed to the texture ( by training on stylized Imagenet) 
I agree with R1 on all accounts (i.e. it is very hard to define the family of biased feature extractors, the proposed approach is ad-hoc, the authors need to compare to texture-shape disentanglement methods, etc), however at the same time, I can see that proposed approach can act as a useful heuristic for regularizing neural networks to pay attention to certain kind of information. 
An interesting use-case of the proposed method (which the authors indirectly mentioned in their response to my review) is in a multi-modal setting. It's not trivial to enforce deep learning systems to utilize all data modalities in a multi-modal setting. By defining G to be models trained on individual modalities, it would be possible to nudge our models to pay attention to the information in all modalities. 
Some important results in deep learning have been ad-hoc (For example skip connections in deep networks, ReLUs) and have nonetheless progressed the field. This work is not as widely applicable as skip connections or ReLUs, but it is, nonetheless, providing a heuristic for solving an important problem.