OK, found it: it had nothing to do with the loss, but with the inputs:
the inputs xs are a list of pretrained word embeddings, and then, I used to build
sentence embeddings before sending to the model training process like that:
xl = []
for xx in xs:
    # sum all words embeddings to get one sentence embedding
    x = torch.sum(xx.squeeze(),dim=0)
    xl.append(x)
# put all sentence embeddings into one tensor
x = torch.stack(xl)

The tensor x is then passed to the standard training process, but I forgot the
x = x.detach()

This used to work with 1.4.0, but not any more with 1.6.0, which makes sense.
Sorry for the false alarm, and thanks again !