  Deep learning models have become popular in the analysis of tabular data, as
they address the limitations of decision trees and enable valuable applications
like semi-supervised learning, online learning, and transfer learning. However,
these deep-learning approaches often encounter a trade-off. On one hand, they
can be computationally expensive when dealing with large-scale or
high-dimensional datasets. On the other hand, they may lack interpretability
and may not be suitable for small-scale datasets. In this study, we propose a
novel interpretable neural network called Neural Classification and Regression
Tree (NCART) to overcome these challenges. NCART is a modified version of
Residual Networks that replaces fully-connected layers with multiple
differentiable oblivious decision trees. By integrating decision trees into the
architecture, NCART maintains its interpretability while benefiting from the
end-to-end capabilities of neural networks. The simplicity of the NCART
architecture makes it well-suited for datasets of varying sizes and reduces
computational costs compared to state-of-the-art deep learning models.
Extensive numerical experiments demonstrate the superior performance of NCART
compared to existing deep learning models, establishing it as a strong
competitor to tree-based models.

Machine Learning