Sravani_Lekkala:

so it would be better to decrease the momentum when we are using small batch size


Well, it won’t hurt, to have say batch_size/momentum constant, when decreasing batch size.



 Sravani_Lekkala:

to improove the accuracy?


I’d rather say that you can sometimes fix batchnorm problems by using smaller momentum, i.e. you can only improve subpar results and only when batchnorm is the issue.