I have read your code, and from my personal perspective, you may adjust your hyper-parameters, network structures etc. Training results depends on your algorithm type, your test scenario, and how you define the rewards.
PS: layer initialization method is also possible to change, since pytorch use xavier to initilize weights, may be consider about using orthorgonal methods.