Hi, I am new in Bert, I want to fine tune a plain text corpus(only text, without any label or other information) to get hidden state layer for specific word/document embedding.
Thus I think the suitable model for my task is the masked  language model, however I found this only example demo    https://huggingface.co/blog/how-to-train 125, which is not universal and clear.
The original examples are here, but no mlm model available https://huggingface.co/transformers/master/custom_datasets.html 62 Does anyone have clear example for this case?
To be specific, I don’t know how to pass true label in MLM task for loss and gradient update. Also while using the Trainer, where should I insert -mlm command?
Thank you in advance