Hello there,
I am interested in training a language model from scratch. Not fine tuning the usual distilbert ---- running the whole thing on my GPU instead !
I found this interesting notebook How to train a new language model from scratch using Transformers and Tokenizers 3 and I would be interested to know if there is one that uses tensorflow instead. I cannot have pytorch on my machine unfortunately.
Is there a huggingface example notebook that would help me do that?
Thanks!