Nice, you’re state sounds great.
Yeah, try accumulating like 100-200 of those episodes’ gradients and average them for the update (the normalizing factor can be included in your learning rate). It’d be great if you let me know what you end up doing, and how it goes!