gpt2 causal mask to skip learning context input? (beginner question)
Suppose we use gpt2 to train a text generation with context or translation with input sequence. 

[google translate<sep> google Übersetzer]. 

Can I ask how we can change the casual mask to let the training only from the token generation after the context input (\<en-de>\)? Like,
[google translate\<en-de\>] to [google]
[google translate\<en-de\> google] to [Übersetzer]

instead of 
[google] to [google translate]

I saw we currently added the casual mask here. 
```
causal_mask = self.bias[:, :, key_length - query_length : key_length, :key_length].bool()
```
https://github.com/huggingface/transformers/blob/b655f16d4eb405d286821bdb68b46b8a989f9c04/src/transformers/models/gpt2/modeling_gpt2.py#L186

Note using the default DataCollatorForLanguageModeling can work for the above task. I am thinking how we can let the training focus on text generation after the context input. 

Can we use the special mask tokens to achieve it? I am a bit doubtful on it. Since it is used for mlm from the code
https://github.com/huggingface/transformers/blob/b655f16d4eb405d286821bdb68b46b8a989f9c04/src/transformers/data/data_collator.py#L350
```
 # If special token mask has been preprocessed, pop it from the dict.
        special_tokens_mask = batch.pop("special_tokens_mask", None)
        if self.mlm:
            batch["input_ids"], batch["labels"] = self.mask_tokens(
                batch["input_ids"], special_tokens_mask=special_tokens_mask
            )
        else:
            labels = batch["input_ids"].clone()
            if self.tokenizer.pad_token_id is not None:
                labels[labels == self.tokenizer.pad_token_id] = -100
            batch["labels"] = labels
        return batch
```

or should we mask all context token input (before \<en-de\>)as -100 in labels so that they would be ignored in the below loss function?
```
            # Shift so that tokens < n predict n
            shift_logits = lm_logits[..., :-1, :].contiguous()
            shift_labels = labels[..., 1:].contiguous()
            # Flatten the tokens
            loss_fct = CrossEntropyLoss()
            loss = loss_fct(shift_logits.view(-1, shift_logits.size(-1)), shift_labels.view(-1))
```

Thanks a lot. cc @patrickvonplaten 