I tried that first but then I got the error:
RuntimeError: Error(s) in loading state_dict for DistributedDataParallel: size mismatch for module.linear.weight: copying a param with shape torch.Size([1000, 1536]) from checkpoint, the shape in current model is torch.Size([1000, 384]).