Solved by JuanFMontesinos in post #4 
              
              
                #SET CONDITIONAL LEARNING RATES IF NECESSARY   
    model_parameters = []
    for n,p in model.named_parameters():
        if n.find('layer_name') != -1:
             model_parameters.append({'params': p, 'lr': LR})

        else:
            model_parameters.append({'params': p, 'lr': LR})

       …