I took the actor-critic example from the examples 80 and turned it into a tutorial with no gym dependencies, simulations running directly in the notebook. I’d like to know if I explained anything poorly or incorrectly or not enough, especially the parts about policy gradients.

  
      github.com
  
  
    spro/practical-pytorch/blob/master/reinforce-gridworld/reinforce-gridworld.ipynb 139
{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](https://i.imgur.com/eBRPvWB.png)\n",
    "\n",
    "# Practical PyTorch: Playing GridWorld with Reinforcement Learning (Policy Gradients with REINFORCE)\n",
    "\n",
    "In this project we'll teach a neural network to navigate through a dangerous grid world.\n",
    "\n",
    "![](http://i.imgur.com/XNGB7sr.gif)\n",
    "\n",
    "Training uses [policy gradients](http://www.scholarpedia.org/article/Policy_gradient_methods) via the REINFORCE algorithm and a simplified Actor-Critic method. A single network calculates both a policy to choose the next action (the actor) and an estimated value of the current state (the critic). Rewards are propagated through the graph with PyTorch's [`reinforce` method](http://pytorch.org/docs/autograd.html?highlight=reinforce#torch.autograd.Variable.reinforce)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},


  This file has been truncated. show original