Hi @BramVanroy, thank you for your answers. As you suggested, one item was not enough to test GPU improvement and I was also taking into account the model loading timing.
Finally, I managed to simplify my code (just in case it helps somebody):
prompt = "In Italy, pizza served in formal settings, such as at a restaurant, is presented unsliced."
next_sentence = "The sky is blue due to the shorter wavelength of blue light."
encoding = tokenizer.encode_plus(prompt, next_sentence, add_special_tokens=True, max_length=tokenizer.max_len, return_tensors='pt').to('cuda:0')
outputs = model(**encoding, next_sentence_label=torch.cuda.LongTensor([1]))
t = outputs.loss
print(t)

Thank you again and sorry for bothering you with this issue, but as I said I am quite new to this transformers “world”