title: Towards General-Purpose Model-Free Reinforcement Learning
summary: Reinforcement learning (RL) promises a framework for near-universal
  problem-solving. In practice however, RL algorithms are often tailored to
  specific benchmarks, relying on carefully tuned hyperparameters and algorithmic
  choices. Recently, powerful model-based RL methods have shown impressive
  general results across benchmarks but come at the cost of increased complexity
  and slow run times, limiting their broader applicability. In this paper, we
  attempt to find a unifying model-free deep RL algorithm that can address a
  diverse class of domains and problem settings. To achieve this, we leverage
  model-based representations that approximately linearize the value function,
  taking advantage of the denser task objectives used by model-based RL while
  avoiding the costs associated with planning or simulated trajectories. We
  evaluate our algorithm, MR.Q, on a variety of common RL benchmarks with a
  single set of hyperparameters and show a competitive performance against
  domain-specific and general baselines, providing a concrete step towards
  building general-purpose model-free deep RL algorithms.
paper_authors._id: 67986cbc7dbf69e4e38539b7
paper_authors.name: Scott Fujimoto
paper_authors._id: 67986cbc7dbf69e4e38539b8
paper_authors.name: Pierluca D'Oro
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T14:09:37.496Z
paper_authors._id: 64b6df54dce8f1fbb8ac9ed7
paper_authors.avatarUrl: /avatars/82ca21cb9c8bacde071769bf4a888375.svg
paper_authors.fullname: Pierluca D'Oro
paper_authors.type: user
paper_authors.user: pierluca
paper_authors._id: 67986cbc7dbf69e4e38539b9
paper_authors.name: Amy Zhang
paper_authors._id: 67986cbc7dbf69e4e38539ba
paper_authors.name: Yuandong Tian
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-01-28T14:10:10.676Z
paper_authors._id: 6344cf73ee1504dbcd5bdfe7
paper_authors.avatarUrl: /avatars/6dd2bf1f9c5679e5c8c85d62c9836aac.svg
paper_authors.fullname: Yuandong Tian
paper_authors.type: user
paper_authors.user: tydsh
paper_authors._id: 67986cbc7dbf69e4e38539bb
paper_authors.name: Michael Rabbat
paper_title: Towards General-Purpose Model-Free Reinforcement Learning
paper_summary: Reinforcement learning (RL) promises a framework for near-universal
  problem-solving. In practice however, RL algorithms are often tailored to
  specific benchmarks, relying on carefully tuned hyperparameters and algorithmic
  choices. Recently, powerful model-based RL methods have shown impressive
  general results across benchmarks but come at the cost of increased complexity
  and slow run times, limiting their broader applicability. In this paper, we
  attempt to find a unifying model-free deep RL algorithm that can address a
  diverse class of domains and problem settings. To achieve this, we leverage
  model-based representations that approximately linearize the value function,
  taking advantage of the denser task objectives used by model-based RL while
  avoiding the costs associated with planning or simulated trajectories. We
  evaluate our algorithm, MR.Q, on a variety of common RL benchmarks with a
  single set of hyperparameters and show a competitive performance against
  domain-specific and general baselines, providing a concrete step towards
  building general-purpose model-free deep RL algorithms.
paper_ai_summary: The paper presents MR.Q, a model-free deep RL algorithm that uses model-based representations to improve performance across diverse benchmarks without incurring high computational costs.
paper_ai_keywords: reinforcement learning model-based RL value function task objectives model-free RL MR.Q