Hi Laurent,
About your observations, this sounds like one of the following:

A regression in the TF2.4.1->TF2.5.0 code.
A change of implementation (e.g. changing a random seed) that will affect individual results, but should not impact global distributions.
A change in API e.g. the default value / logic of a method has been changed.

Case 2 can be validated or eliminated by replicating the experiment multiple times: If the results are 50% of the time better, case 2 is likely true. If the results are always worse, case 2 is likely false.
For case 1 and 3, it might be interesting to create and share a minimal colab using public data that illustrates the problem. If the complexity of the model is small, it might be possible to detect case 3. Otherwise, this is the kind of colab that should be reported as an issue to TF. However, there is no guarantee of a quick resolution.
A few things have changed internally between TF2.4.1 and TF2.5.0, and we tried not to have TF-DF for both versions. It is possible to recompile TF-DF for TF2.4.1 with a few tweaks, but it is probably not interesting in your case. Sorry I cannot help you more.
Cheers,
M.