Hi Guys,
how do i further train a Model someone else already fine-tuned? I wanted to load a already Fine-Tuned Modell like this:
    model = Wav2Vec2ForCTC.from_pretrained(
        model_args.model_name_or_path,
        cache_dir=model_args.cache_dir,
        activation_dropout=model_args.activation_dropout,
        attention_dropout=model_args.attention_dropout,
        hidden_dropout=model_args.hidden_dropout,
        feat_proj_dropout=model_args.feat_proj_dropout,
        mask_time_prob=model_args.mask_time_prob,
        gradient_checkpointing=model_args.gradient_checkpointing,
        layerdrop=model_args.layerdrop,
        ctc_loss_reduction="mean",
        pad_token_id=processor.tokenizer.pad_token_id,
        vocab_size=len(processor.tokenizer),
    )

But im getting:
RuntimeError: Error(s) in loading state_dict for Wav2Vec2ForCTC:
	size mismatch for lm_head.weight: copying a param with shape torch.Size([174, 1024]) from checkpoint, the shape in current model is torch.Size([35, 1024]).
	size mismatch for lm_head.bias: copying a param with shape torch.Size([174]) from checkpoint, the shape in current model is torch.Size([35]).

How would i do this right?
Something like
    model = Wav2Vec2ForCTC.from_pretrained(model_args.model_name_or_path)
    model.to("cuda")
    model.freeze_feature_extractor()
    processor = Wav2Vec2Processor.from_pretrained(model_args.model_name_or_path)

But im not sure if this is the way to go.
Ty in advanced