How to update the GPT2 with loss which are provided from another separate module?
Suppose I have N prompts(sentences) for generation. They are fed into GPT2 and get the corresponding synthesis sentences. 
And I have a separate black box which can return loss given these synthesis samples. The black box is just another component. 
It is natural for every batch that GPT2 generate samples and get the loss, repeatedly.
What I want to do is use the loss from the black box to update the parameters of GPT2, at each batch.
The generation of GPT2 is quite simple, but how can I implement the idea of updating it with the loss?
Is there any example for doing this ?
Please give some thoughts, thanks.
I guess you can do it like so:
```
from transformers import GPT2Tokenizer, GPT2LMHeadModel
tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
model = GPT2LMHeadModel.from_pretrained('gpt2')

model.train()
# when generating, we will use the logits of right-most token to predict the next token
# so the padding should be on the left
tokenizer.padding_side = "left" 
tokenizer.pad_token = tokenizer.eos_token # to avoid an error

prompts = ["Hello, my dog is a little", "Hello, my dog is"]
inputs = tokenizer(prompts, padding=True, return_tensors="pt")

output_sequences = model.generate(
    input_ids=inputs['input_ids'],
    attention_mask=inputs['attention_mask']
)

loss = black_box(output_sequences)
loss.backward()
```

Please note that the [forum](https://discuss.huggingface.co/) is a better place to ask questions, Github issues are mostly for bugs/feature requests.

Thanks.<|||||>> I guess you can do it like so:
> 
> ```
> from transformers import GPT2Tokenizer, GPT2LMHeadModel
> tokenizer = GPT2Tokenizer.from_pretrained('gpt2')
> model = GPT2LMHeadModel.from_pretrained('gpt2')
> 
> model.train()
> # when generating, we will use the logits of right-most token to predict the next token
> # so the padding should be on the left
> tokenizer.padding_side = "left" 
> tokenizer.pad_token = tokenizer.eos_token # to avoid an error
> 
> prompts = ["Hello, my dog is a little", "Hello, my dog is"]
> inputs = tokenizer(prompts, padding=True, return_tensors="pt")
> 
> output_sequences = model.generate(
>     input_ids=inputs['input_ids'],
>     attention_mask=inputs['attention_mask']
> )
> 
> loss = black_box(output_sequences)
> loss.backward()
> ```
> 
> Please note that the [forum](https://discuss.huggingface.co/) is a better place to ask questions, Github issues are mostly for bugs/feature requests.
> 
> Thanks.

thanks, very helpful. 