Ah yes, if possible I think you’d be better off using the BertForSequenceClassification class together with BertConfig instead of the custom class you created.
For example, you might be able to make this work as follows:
config = ...
model = BertForSequenceClassification.from_pretrained(destination_folder+'model.pt', config=config)

and then passing model and the tokenizer to the pipeline as before. If not, then you’ll probably have to re-train the model or live with the default labels from the pipeline