hi dave-kudo,
[I am not an expert]
I think that would be expected.  If you ran fine-tuning with your new corpus as well as the original training corpus, then your small corpus would have little impact.   However, now that you are training only with your small corpus, the model “thinks” that the new corpus is the whole of its training data.   It doesn’t “know” that it is already very well trained (to the original data).   It puts all its effort into optimising for the new corpus, including forgetting a lot of earlier stuff.
Have you tried using a smaller learning-rate?
Another possibility is to freeze several of the early layers of the model, so that your fine-tuning is only affecting the last few layers.