Thanks for the reply. I’ve checked their code but it seems that they only implemented for inference, while I was trying to get a full estimation of FLOPs on training. But I guess just as tymokvo said, without knowing how the batchnorm is implemented in cuDNN, I can’t make any reasonable estimation.