I think the expectation wouldn’t hold, since the model could learn to extract features only for the current task.
While the weights for the 5 previously classes in the classification layer wouldn’t be updated, the feature extractor would.
I would assume the model to forget the previously learned classes especially if the features between the tasks differ a lot.
E.g. an artificial examples would be to try to learn to classify 5 bird classes first and 5 cancer classes based on CT scans later. The model could try to extract a completely different feature set for these images and I would expect it to decrease its performance on the bird dataset.
Of course, your use case is different, i.e. your samples come from the same dataset, but I would still think the model could focus on different feature sets.
Do you have any resources which have evaluated the ability to “remember a task” of DL models?