Hi @dikster99,

So, to get me started with the MultiClass task where I have 1 label but 6 values (VeryPositiv, Positiv, No_Opinion, Mixed_Opinion, Negativ, VeryNegativ) - I could actually use, for example:

a model trained for a True/False sentiment,
specify num_labels=1 and train it with my labels containing 6 values
and that should work if my labels are sensible.

Did I get that right?

I think in this case you actually have 6 labels, one for each independent category. For example, such a dataset might take the following form:




Text
Label




I really   Transformers!
VeryPositiv


The pandemic is a drag 

Negativ


I went for a walk today
No_Opinion


…
…



so you have one label column, but each row can take one of six values.
To get started, my suggestion would be to load up one of the pretrained language models (e.g. distilbert-base-uncased), specify num_labels=6, and then tokenize the dataset / build the Trainer as done in Sylvain’s tutorial, e.g. something like
model_ckpt = "distilbert-base-uncased"
tokenizer = AutoTokenizer.from_pretrained(model_ckpt)
model = AutoModelForSequenceClassification.from_pretrained(model_ckpt, num_labels=6).to("cuda")

def tokenize_text(batch):
    # Replace "text" with whatever column name has your text inputs
    return tokenizer(batch["text"], truncation=True)

dataset_enc = dataset.map(tokenize_text, batched=True)
...

Regarding, your experience with downloading the models locally, did you know about the “Use in transformers” button from the model hub 3?

Screen Shot 2021-02-28 at 9.15.59 pm1550×1112 129 KB

If not, perhaps it should be made clearer in the documentation but for that I suggest opening an issue or PR on GitHub so the maintainers can provide you feedback