What is domain adaptation? When fine-tuning a pretrained model on a new dataset, the fine-tuned model we obtain will make predictions that are attuned to this new dataset. When the two models are trained with the same task, we can then compare their predictions on the same input. The predictions of the two models will be different, in a way that reflects the differences between the two datasets, a phenomenon we call domain adaptation. Let's look at an example with mask language modeling, by comparing the outputs of the pretrained distilBERT model with the version fine-tuned in chapter 7 of the course (linked below). The pretrained model makes generic predictions, whereas the fine-tuned model has its first two predictions linked to cinema. Since it was fine-tuned on a movie reviews dataset, it's perfectly normal to see it adapted its suggestions like this. Notice how it keeps the same predictions as the pretrained model afterward. Even if the fine-tuned model adapts to the new dataset, it's not forgetting what it was pretrained on. This is another example on a translation task. On top we use a pretrained French/English model and at the bottom, the version we fine-tuned in chapter 7. The top model is pretrained on lots of texts, and leaves technical English terms like plugin and email unchanged in the translation (both are perfectly understood by French people). The dataset picked for the fine-tuning is a dataset of technical texts where special attention was picked to translate everything in French. As a result, the fine-tuned model picked that habit and translated both plugin and email.