This awful or least it seems to be less than stable 
So, I tried to apply the code that we already had working for English text just to find out that it does not work for German - because there are not many German public datasets around I created my own 1.
I then published 3 notebooks on GitHub to show the dilemma:


66_Multi_Label_German_text_classification_in_TensorFlow_Keras.ipynb 13
Shows a Multilabel approach via a classic Keras/TF implementation to show that the German dataset can yield useful results (the results are not great because I did not clean the data extracted originally somewhere else but this shows the dataset can be used for a classification task)


66_Transformer_4_Language_Classification_MultiLabel_DistilBert.ipynb 50
Shows a Multilabel approach via Transformers DistillBert Modeling using the English Toxitity dataset collection as we previously discussed.
This model is working as expecting and does indeed return useful results.


66_Transformer_4_Language_Classification_MultiLabel_DistilBert_German.ipynb 14
This notebook uses the exact same transformers code approach as we’ve used in 2) with the difference that we try to consume German data here. The accuracy shown during the training phase looks useful but testing at the end of the notebook shows that the actual predictions are less than useful (no single prediction is correct and the model always predicts very similar output no matter what the input is)


I wonder if this is a known bug or whether this should be an issue for the HuggingFace repository?