Thanks for the reply., should I do this for every layer? For example if the above architecture is part of a model class.
for epoch in number_of_epochs:
    if epoch < 10 : 
         for p in model.input.parameters():
             p.requires_grad = True
         for q in model.layer1.parameters():
             q.requires_grad = True
   else: 
         for p in model.input.parameters():
             p.requires_grad = False
         for q in model.layer1.parameters():
             q.requires_grad = False

would this thing work?