Answers.Body: <p>Think of the network as a feature extractor followed by a policy head and a value function head. The feature extractor compresses the inputs into a lower dimensional feature vector that we hypothesize will be useful to both the policy and the value function. Then you train the policy and value function with those learned features as inputs (which is in theory much easier than training both with higher dimensional inputs). This way you save a bunch of parameters and hopefully accelerate training.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-06-04T14:02:59.377Z
Answers.LastActivityDate: 2021-06-04T14:02:59.377Z
Answers.UserDisplayName: harwiltz
Body: <p>The PPO objective may include a value function error term when parameters are shared between the policy and value functions. How does this help, and when to use a neural network architecture that shares parameters between the policy and value functions, as opposed to two neural networks with separate parameters?</p>
  
Tags: <reinforcement-learning><deep-rl><proximal-policy-optimization>
Title: How does sharing parameters between the policy and value functions help in PPO?
UserDisplayName: Mika
ThreadText: # How does sharing parameters between the policy and value functions help in PPO?
  
  (Asked by: username_0 on 2021-05-23)
  
  The PPO objective may include a value function error term when parameters are shared between the policy and value functions. How does this help, and when to use a neural network architecture that shares parameters between the policy and value functions, as opposed to two neural networks with separate parameters?
  
  **Tags:** reinforcement-learning, deep-rl, proximal-policy-optimization
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-06-04. Score: 2
  
  Think of the network as a feature extractor followed by a policy head and a value function head. The feature extractor compresses the inputs into a lower dimensional feature vector that we hypothesize will be useful to both the policy and the value function. Then you train the policy and value function with those learned features as inputs (which is in theory much easier than training both with higher dimensional inputs). This way you save a bunch of parameters and hopefully accelerate training.
  
  
  ---
  
  