Solved by ptrblck in post #2 
              
              
                The issue is created by the inplace unsqueeze_ call on action_value, but is raised in the tanh. 
If you use action_value = action_value.unsqueeze(-1) instead, your code should work.