I think you should look at this topic explaining the tensor.reinforce method:

  
    
    
    What is action.reinforce(r) doing actually? reinforcement-learning
  
  
    Hi, 
I am studying RL with reinforcement/reinforce.py in pytorch/examples. I have some questions about it. 


What does action.reinforce® internally do ? 


Below is REINFORCE update rule where v_t is a return. We need to do gradient “ascent” as below but if we use optimizer.step, it is gradient “descent”. Is [action.reinforce®]((https://github.com/pytorch/examples/blob/master/reinforcement_learning/reinforce.py) multiplying log probability by -r? Then, it makes sense. 
[image] 


In autograd.ba…
  


If the action is the result of a sampling, calling action.reinforce(r) acts as a policy gradient.
You can find a code example of implementation here:

  
      github.com
  
  
    pytorch/examples/blob/master/reinforcement_learning/reinforce.py 1.5k
import argparse
import gym
import numpy as np
from itertools import count

import torch
import torch.nn as nn
import torch.nn.functional as F
import torch.optim as optim
from torch.distributions import Categorical


parser = argparse.ArgumentParser(description='PyTorch REINFORCE example')
parser.add_argument('--gamma', type=float, default=0.99, metavar='G',
                    help='discount factor (default: 0.99)')
parser.add_argument('--seed', type=int, default=543, metavar='N',
                    help='random seed (default: 543)')
parser.add_argument('--render', action='store_true',
                    help='render the environment')
parser.add_argument('--log-interval', type=int, default=10, metavar='N',


  This file has been truncated. show original