Well, I came back to this after a few days, and I’m still stuck. So any advice will make you a genius in my view?
Here’s a post of my code as simple as I could make it as a big blob,

  
      gist.github.com
  
  
    https://gist.github.com/AjayTalati/184fec867380f6fa22b9aa0951143dec 117
main_single.py
# Reference - Deepmind A3C's paper, https://arxiv.org/pdf/1602.01783.pdf
# See section 9 - Continuous Action Control Using the MuJoCo Physics Simulator
#
# Code based on https://github.com/pfre00/a3c

import argparse
import torch
import torch.multiprocessing as mp
import gym
import torch.nn as nn
This file has been truncated. show original




  
  
    
    
  
  


I keep getting this error,
File "main_single.py", line 174, in <module>
value_loss = value_loss + advantage.pow(2)
AttributeError: 'numpy.ndarray' object has no attribute 'pow'

I don’t understand why advantage has become a numpy.array instead of a torch.tensor - it never occurred with the discrete action implementation?
Any ideas what I’ve got wrong?
Thanks a lot for your help,
Best,
Ajay