I’m using Encoder-Decoder model to train a translation task, while partial of the data are unlabeled.
For labeled data, I can use the following codes to do the inference and compute the loss,
# model is composed of EncoderDecoder architecture
# source_data and target_data are processed by tokenizer beforehand
batch = {
    "inputs_idx": source_data["inputs_idx"],
    "attention_mask": source_data["attention_mask"],
    "decoder_input_ids": target_data["inputs_idx"]
    "decoder_attention_mask": target_data["attention_mask"],
    "labels": target_data["inputs_idx"].clone()
}

output = model(**batch)
supervised_loss = output["loss"]

Besides the supervised loss, I also want to compute some unlabeled loss over the predicted logits of unlabeled source data, such as,
batch = {
    "inputs_idx": source_data["inputs_idx"],
    "attention_mask": source_data["attention_mask"],
}

output = model(**batch)

unsupervised_loss = some_loss_func(output["logits"])

However, I can not do the inference without specifying “decoder_input_ids”, the decoder will produce error about
You have to specify either input_ids or inputs_embeds

So far, I assign source_data["idx"] for decoder_input_ids to avoid the issue, but I feel like it is incorrect cause it will bring inconsistency in inference between labeled and unlabeled data. So, I am wondering how should I do inference for unlabeled data correctly.