Hi! I think the preprocessing makes sense, but not very well, as the difference of the two frames capture the following information:

the current position and the next position
which position is current, and which position is next

However, it does not capture the absolute position very well, because it uses a region of interest, which will keep fixed when the cartpole is in the middle. I think this might be where divergence comes.