Instead of modifying the optimizer like that, a better way of doing it would be:
for param in model_gpt2.transformer.h.parameters():
    param.requires_grad = False
for param in model_gpt2.multiple_choice_head.parameters():
    param.requires_grad = True