Solved by LeviViana in post #2 
              
              
                Doesn’t torch.nn.BatchNorm fit your request? Otherwise, you can always break your forward method into many other methods to get the intermediate results, and normalize them as you wish. The most complex case could come from modules that you didn’t implement yourself, in that case, you can always ove…