PPO is tricky one to fine tune. For me it was a lot of trial and error to get it work on my own implementation 20. You may take a look at the hyperparameters I used for some OpenAI gym problems, maybe it will work for you too. You may also want to try normalize the state if it is complex on the harder problems. I hope it helps