How do you keep the average over 200 for a period? In my experiments, it cannot keep. There is only bursts into high durations and the following duration will fall to very low values. I also tried the the update frequency but it does not work. Can you show me your duration-episode curve?
I am also using the difference image as the state.