The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forest
<|reference_start|>The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forest: Random forests are classical ensemble algorithms that construct multiple randomized decision trees and aggregate their predictions using naive averaging. \citet{zhou2019deep} further propose a deep forest algorithm with multi-layer forests, which outperforms random forests in various tasks. The performance of deep forests is related to three hyperparameters in practice: depth, width, and tree size, but little has been known about its theoretical explanation. This work provides the first upper and lower bounds on the approximation complexity of deep forests concerning the three hyperparameters. Our results confirm the distinctive role of depth, which can exponentially enhance the expressiveness of deep forests compared with width and tree size. Experiments confirm the theoretical findings.<|reference_end|>
@article{lyu2024the,
    title={The Role of Depth, Width, and Tree Size in Expressiveness of Deep Forest},
    author={Shen-Huan Lyu, Jin-Hui Wu, Qin-Cheng Zheng, Baoliu Ye},
    journal={In: Proceedings of the 27th European Conference on Artificial
  Intelligence, 2024},
    year={2024},
    archivePrefix={arXiv},
    eprint={2407.05108},
    primaryClass={cs.LG stat.ML}
}