Hi @aleemsidra
The learning rate scheduler can only be applied during the learning phase, i.e. during training. No learning is done during the validation / evaluation phase, so it doesn’t make sense speaking about LR scheduler here. All that happens during the validation phase is that the model uses the weights it has already learned during training to do some predictions, and compares them with the expected / actual values. The model’s weights do not change during validation.
During training you can use the LR scheduler if you want your LR to change, as opposed to being constant. This is because in many cases, reducing the learning rate and making the optimiser steps smaller and smaller as you’re approaching the loss function minimum can be beneficial, however this depends on your dataset really.
You can apply the LR scheduler after each mini-batch if you want, or you could choose to change your learning rate once per epoch. That depends on your use case and model configuration, in large NLP transformer models I often apply the LR scheduler step after every minibatch. If you have a specific model / use case in mind, then let us know, and people specialised in that specific area would be able to give more specific advice