title: NRGBoost: Energy-Based Generative Boosted Trees
summary: Despite the rise to dominance of deep learning in unstructured data domains,
  tree-based methods such as Random Forests (RF) and Gradient Boosted Decision
  Trees (GBDT) are still the workhorses for handling discriminative tasks on
  tabular data. We explore generative extensions of these popular algorithms with
  a focus on explicitly modeling the data density (up to a normalization
  constant), thus enabling other applications besides sampling. As our main
  contribution we propose an energy-based generative boosting algorithm that is
  analogous to the second order boosting implemented in popular packages like
  XGBoost. We show that, despite producing a generative model capable of handling
  inference tasks over any input variable, our proposed algorithm can achieve
  similar discriminative performance to GBDT on a number of real world tabular
  datasets, outperforming alternative generative approaches. At the same time, we
  show that it is also competitive with neural network based models for sampling.
paper_authors._id: 67039db4dae57ebb932171ec
paper_authors.name: João Bravo
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2024-10-07T09:30:23.258Z
paper_authors._id: 67039d3a8790e37e2e31c58d
paper_authors.avatarUrl: /avatars/809e57f523b78ca5498ff8001f135918.svg
paper_authors.fullname: João Bravo
paper_authors.type: user
paper_authors.user: joaobravo
paper_title: NRGBoost: Energy-Based Generative Boosted Trees
paper_summary: Despite the rise to dominance of deep learning in unstructured data domains,
  tree-based methods such as Random Forests (RF) and Gradient Boosted Decision
  Trees (GBDT) are still the workhorses for handling discriminative tasks on
  tabular data. We explore generative extensions of these popular algorithms with
  a focus on explicitly modeling the data density (up to a normalization
  constant), thus enabling other applications besides sampling. As our main
  contribution we propose an energy-based generative boosting algorithm that is
  analogous to the second order boosting implemented in popular packages like
  XGBoost. We show that, despite producing a generative model capable of handling
  inference tasks over any input variable, our proposed algorithm can achieve
  similar discriminative performance to GBDT on a number of real world tabular
  datasets, outperforming alternative generative approaches. At the same time, we
  show that it is also competitive with neural network based models for sampling.
paper_ai_summary: Energy-based generative boosting achieves competitive discriminative and generative performance on tabular data, outperforming existing generative approaches and matching neural network performance in sampling.
paper_ai_keywords: Random Forests Gradient Boosted Decision Trees generative extensions data density energy-based generative boosting second order boosting XGBoost