ptrblck:

will thus use the batch stats to normalize the input not the running stats.


I think I got your point…!
In the training mode, we always use the batch statistics for normalization, but in the meantime the running statistics are updated w.r.t. the batch statistics.
However, it the CNN is set to the evaluation mode, only the running statistics are used for both training and testing stages.
Hmmmmmm may I know if my understand is correct?
Thank you.