Hi,
I eventually realised this was an issue due to an outdated transformer version I updated and then decided to write my own training loop. I now have a model which is training.
Thanks for your help
A