Hi @bas!
Actually, while there is a bit of an issue here, there is nothing for you to worry about.
If you want the full explanation, you’re basically getting two messages here:
Some layers from the model checkpoint at distilbert-base-uncased-finetuned-sst-2-english were not used when initializing TFDistilBertModel: ['pre_classifier', 'classifier', 'dropout_19']

and
Some layers from the model checkpoint at distilbert-base-uncased-finetuned-sst-2-english were not used when initializing TFDistilBertForSequenceClassification: ['dropout_19']

You shouldn’t be getting the first one, as it tells you that it is loading the checkpoint in the TFDistilBertModel and that there are some missing weights. The first one happens because we’re doing a sanity check to validate that the checkpoint can be safely loaded in that framework (here that’s TensorFlow). We should have a better way of doing this, as we’re not interested in the TFDistilBertModel but the TFDistilBertForSequenceClassification. We shouldn’t need to load the weights in the first model, and we definitely shouldn’t output missing weights/unused weights for that step.
Regarding the second one, it tells you that the TFDistilBertForSequenceClassification does not use dropout_19 in its architecture, and, therefore, doesn’t load it. However, it newly initializes dropout_38. You shouldn’t worry about this either, because dropouts have no weights. We should probably do a check and only add to unused weights/newly initialized weights when the layers are not dropouts.