it’s not clear what your robot environment’s action space.
With cartpole the action space is discrete action selecting either left and right. (ie. your policy selects either 0 or 1 for left or right action).
What is the action space for your robotic example? I see in the rep the action space is 2*n, where n is number of joints.
Is your policy suppose to output real values (for example degrees of rotation) for each joint? Or is it just suppose to pick 1 joint from the 2*n joints available?
If you’re using the same cartpole implementation for policy, then it will only work for the second case I described above.
If your policy is suppose to output real joint angle values, then you will need a continuous policy distribution (i.e pytorch.distributions.normal) for each k joint angles.