@kmysiak About your question, my understanding is that if you load a model using from_pretrained method given either a model name or path to your checkpoint, the model will be trained starting from that checkpoint. One thing to keep in mind (you may already know this) is that there is also a possibility that your further pretraining causes the model to forget some of the knowledge it already has. That is something you need to test and run experiments to figure out.