Thanks. I agree that it is very likely that nn.Conv2d works that way. Maybe the case I give is not general for other autograd functions. Anyway, I still hope that someone can give a more solid explanation.