About chain rule, gradients of input transformations wrt input should not be ignored with freezing, i.e. any module can be viewed as some chainable transformation f(x,p) (p - local parameters) and freezing implies ignoring p gradients: f(x,p.detach())