Hi,
I’m getting back to this now. Any luck with NSP? Are you using tensorflow or pytorch? Did you try adding learning rate scheduler to the optimizer such as PolynomialDecay? Fine-tuning a pretrained model - Hugging Face Course 13