title: Gradient Boosting Reinforcement Learning
summary: Neural networks (NN) achieve remarkable results in various tasks, but lack
  key characteristics: interpretability, support for categorical features, and
  lightweight implementations suitable for edge devices. While ongoing efforts
  aim to address these challenges, Gradient Boosting Trees (GBT) inherently meet
  these requirements. As a result, GBTs have become the go-to method for
  supervised learning tasks in many real-world applications and competitions.
  However, their application in online learning scenarios, notably in
  reinforcement learning (RL), has been limited. In this work, we bridge this gap
  by introducing Gradient-Boosting RL (GBRL), a framework that extends the
  advantages of GBT to the RL domain. Using the GBRL framework, we implement
  various actor-critic algorithms and compare their performance with their NN
  counterparts. Inspired by shared backbones in NN we introduce a tree-sharing
  approach for policy and value functions with distinct learning rates, enhancing
  learning efficiency over millions of interactions. GBRL achieves competitive
  performance across a diverse array of tasks, excelling in domains with
  structured or categorical features. Additionally, we present a
  high-performance, GPU-accelerated implementation that integrates seamlessly
  with widely-used RL libraries (available at https://github.com/NVlabs/gbrl).
  GBRL expands the toolkit for RL practitioners, demonstrating the viability and
  promise of GBT within the RL paradigm, particularly in domains characterized by
  structured or categorical features.
paper_authors._id: 66908cf75cb4c07caa1f91c2
paper_authors.name: Benjamin Fuhrer
paper_authors.status: extracted_pending
paper_authors.statusLastChangedAt: 2024-07-12T01:55:08.622Z
paper_authors._id: 64a58d68f95d48e7c521008d
paper_authors.avatarUrl: /avatars/64fdf06190064623e16fadbc3965a847.svg
paper_authors.fullname: benjamin fuhrer
paper_authors.type: user
paper_authors.user: bfuhrer
paper_authors._id: 66908cf75cb4c07caa1f91c3
paper_authors.name: Chen Tessler
paper_authors.status: extracted_confirmed
paper_authors.statusLastChangedAt: 2025-03-18T21:31:51.065Z
paper_authors._id: 67d9db8a8d942645bf063484
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/7JeFDO4Ky7XjTCRL4ROBl.png
paper_authors.fullname: Chen Tessler
paper_authors.type: user
paper_authors.user: ctessler
paper_authors._id: 66908cf75cb4c07caa1f91c4
paper_authors.name: Gal Dalal
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-07-15T13:34:43.925Z
paper_authors._id: 66952220948a73be7f51d4ff
paper_authors.avatarUrl: /avatars/1f6cfe7502b2cb697816caa3c5334c7b.svg
paper_authors.fullname: Gal Dalal
paper_authors.type: user
paper_authors.user: GalDalal
paper_title: Gradient Boosting Reinforcement Learning
paper_summary: Neural networks (NN) achieve remarkable results in various tasks, but lack
  key characteristics: interpretability, support for categorical features, and
  lightweight implementations suitable for edge devices. While ongoing efforts
  aim to address these challenges, Gradient Boosting Trees (GBT) inherently meet
  these requirements. As a result, GBTs have become the go-to method for
  supervised learning tasks in many real-world applications and competitions.
  However, their application in online learning scenarios, notably in
  reinforcement learning (RL), has been limited. In this work, we bridge this gap
  by introducing Gradient-Boosting RL (GBRL), a framework that extends the
  advantages of GBT to the RL domain. Using the GBRL framework, we implement
  various actor-critic algorithms and compare their performance with their NN
  counterparts. Inspired by shared backbones in NN we introduce a tree-sharing
  approach for policy and value functions with distinct learning rates, enhancing
  learning efficiency over millions of interactions. GBRL achieves competitive
  performance across a diverse array of tasks, excelling in domains with
  structured or categorical features. Additionally, we present a
  high-performance, GPU-accelerated implementation that integrates seamlessly
  with widely-used RL libraries (available at https://github.com/NVlabs/gbrl).
  GBRL expands the toolkit for RL practitioners, demonstrating the viability and
  promise of GBT within the RL paradigm, particularly in domains characterized by
  structured or categorical features.
paper_ai_keywords: Gradient Boosting Trees (GBT) Gradient-Boosting RL (GBRL) actor-critic algorithms tree-sharing approach policy and value functions distinct learning rates GPU-accelerated implementation