Have you tried to set the eps argument of your ReduceLROnPlateau scheduler?
Adam’s eps is different to the one used in the scheduler.
From the docs:

eps ( float ) – Minimal decay applied to lr. If the difference between new and old lr is smaller than eps, the update is ignored. Default: 1e-8.

If you set it to a smaller value, the scheduler should continue decreasing the learning rate.