Automatically feature generation is a major topic of automated machine learning. Among various feature generation approaches, feature crossing, which takes cross-product of sparse features, is a promising way to effectively capture the interactions among categorical features in tabular data. Previous works on feature crossing try to search in the set of all the possible cross feature fields. This is obviously not efficient when the size of original feature fields is large. Meanwhile, some deep learning-based methods combines deep neural networks and various interaction components. However, due to the existing of Deep Neural Networks (DNN), only a few cross features can be explicitly generated by the interaction components. Recently, piece-wise interpretation of DNN has been widely studied, and the piece-wise interpretations are usually inconsistent in different samples. Inspired by this, we give a definition of interpretation inconsistency in DNN, and propose a novel method called CrossGO, which selects useful cross features according to the interpretation inconsistency. The whole process of learning feature crossing can be done via simply training a DNN model and a logistic regression (LR) model. CrossGO can generate compact candidate set of cross feature fields, and promote the efficiency of searching. Extensive experiments have been conducted on several real-world datasets. Cross features generated by CrossGO can empower a simple LR model achieving approximate or even better performances comparing with complex DNN models.
The paper presents a scheme to generate new features as cross-product
of binary features to improve the performance of linear models while
obtaining interpretable models. The candidate set of cross-features
can be exponential and is handled by the proposed scheme by utilizing
the gradient-based importances of the features in a (deep) neural
network. Features with large discrepancies in their local and global
interpretations are used as the seed set of candidate features for
generating new cross-features, and the final step performs a feature
selection to further reduce the final set of cross-features. The
empirical evaluation demonstrates the utility of the proposed scheme
on 8 datasets. 
While the proposed scheme does present a way to improve the accuracy
of interpretable models, I am currently recommending a reject for the
following reasons (given the higher standard recommended for papers
over 8 pages):
- While this paper does consider some baselines, it seems to be
  missing some crucial baselines that address the same (or very
  similar) problem. There are some papers [2,3] that learn boolean
  conjunctions (that can be seen as cross-features) to generate
  accurate interpretable models. Moreover, there are some search based
  feature generation schemes [1,4] that significantly improve upon the
  exhaustive feature generation scheme of Kanter & Veeramacheneni,
  2015. This technique can easily be applicable in learning boolean
  cross features with binary features. At the very least, it is important to 
  understand where this proposed scheme is positioned relative to the 
  aforementioned literature and why a comparison is not required.
- It is very unintuitive (at least to me) to tie the candidate
  generation scheme to a neural network especially given the
  sensitivity of neural network training to different initializations
  and other factors. For the same data and neural network, the local
  vs. global discrepancies can change significantly, thereby changing
  the candidate set of cross features. This can potentially make the
  proposed feature generation scheme somewhat unstable, and the
  interpretations from the subsequent models might not be as
  interpretable as they seem. It would be good to understand what I am
  missing here and why being tied to a neural network model is
  essential and not an issue here. 
Clarification:
- Lines 8-10 in Algorithm 1 is not clearly explained.
- The experiment to motivate Assumption 1 needs to be better
  explained.
Minor:
- The notation in equation (1) needs to be clarified better.
[1] Khurana, Udayan, et al. "Cognito: Automated feature engineering
for supervised learning." 2016 IEEE 16th International Conference on
Data Mining Workshops (ICDMW). IEEE, 2016. 
[2] Dash, Sanjeeb, Oktay Gunluk, and Dennis Wei. "Boolean decision
rules via column generation." Advances in Neural Information
Processing Systems. 2018. 
[3] Wei, Dennis, et al. "Generalized Linear Rule Models." Proceedings
of the 36th International Conference on Machine Learning. 2019.
[4] Khurana, Udayan, Horst Samulowitz, and Deepak Turaga. "Feature
engineering for predictive modeling using reinforcement learning."
Thirty-Second AAAI Conference on Artificial Intelligence. 2018.