You can use it with a pretrained model in any language, but you would need your pretrained model to be in the same language as the dataset you are using (otherwise, the texts seen won’t make sense to the model!)