Hi, @ptrblck, thank you for your great answer. But may I ask why BN layer will change the model behavior? To on the same page, I am referring the following case:
model.train()
output1 = model(input)
output2 = model(input)

Will output1 and output2 be different?