@ptrblck
I want to load dataset 1 with freeze 6-11 layers and load other dataset2 with freeze 0-5 layers my code for one dataset1:
for epoch in range(EPOCHS):
       print(f'Epoch {epoch + 1}/{EPOCHS}')
       print('-' * 10)
       if epoch==0:
          for name, param in list(xlnet_model.named_parameters())[104:206]: 
                    print('I will be frozen: {}'.format(name))
                    #print(param)
                    param.requires_grad = False
       

       train_acc, train_loss = training(xlnet_model,train_data_loader,optimizer, device, scheduler, len(train_data))

       print(f'Train loss {train_loss} Train accuracy {train_acc}')

will this statement  "for name, param in list(xlnet_model.named_parameters())[104:206]: " freeze only 6-11 layer and othe weight will update?( i am using pertrained model)