The patience is applied to the last minimal loss value and the subsequent values.
Let’s analyze the behavior for patience=0:

Until epoch10 the loss is decreasing (starting with epoch0).
The loss in epoch11 increases; since patience=0, we are decreasing the lr. The current min value is 10980 from epoch9.
The loss decreases to 7322 in epoch 12, which is our new minimum
After it the loss increases in every epoch and the lr is also decreased in every epoch until eps is met.

Let’s now have a look at patience=5:

Again the loss decreases until epoch10=10980, which is our current minimum
The loss increases in epoch11; our “current” patience is now 4; the current minimum stays the same.
Epoch 12 yields a new minimum of 7322. Patience is back to 5.
The loss increases in the next 4 epochs. Patience is lowering to 4, 3, 2, 1.
Epoch17 yields a new minimum of 5167. Patience is back to 5.
The loss increases for the rest of the epoch. After 5 increases, the “current” patience is at 0 and the lr will be decreased.