Solved by I-Love-U in post #2 
              
              
                Maybe you should load the parameters like this: 
def load_pretrained_params(model, model_state_path: str):
    pretrained_dict = torch.load(model_state_path, map_location="cpu")
    model_dict = model.state_dict()
    # 1. filter out unnecessary keys
    if list(pretrained_dict.keys())[0].startswith…