Solved by Brando_Miranda in post #7 
              
              
                I think this will work for you, just change it to your custom layer. Let us know if did work: 
def replace_bn(module, name):
    '''
    Recursively put desired batch norm in nn.module module.

    set module = net to start code.
    '''
    # go through all attributes of module nn.module (e.g. netw…