Hi all,
yesterday through a workshop I learned about this forum. So I have the following question: is it possible to further pre-train transformers (e.g. BERT, DistilBert) using my own corpus? I mean not for the downstream task, but the language model (e.g. BERT tasks MSM and NSP) itself? Is it possible in general and with huggingface specifically?
Thank you. best regards
LIza