How to enable grad_fn when calling the generate() method of a T5 model
I was trying to attribute a prediction by a T5 model to the words of an input by gradient method. For a ```T5_model```, I call ```T5_model(**inputs)``` when training, and call ```T5_model.generate(**inputs)``` when doing inference. In training, the grad_fn is enabled for the loss but not in inference.

So how do I enable grad_fn when calling the ```generate()``` method to generate a prediction? So that I can get the gradient of the prediction with respect to each word of the input sentence.
In fact, in the implementation file of T5, [modeling_t5.py](https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_t5.py), I did not see ```with torch.no_grad:```. So I was wondering how ```grad_fn``` is disabled in the ```generate()``` method.<|||||>Although I think it has a low chance to work, I tried to add ```torch.set_grad_enabled(True)``` before calling the ```generate()``` method, but the ```grad_fn``` is still not enabled.<|||||>It turned out there is a decorator before the [generate() method](https://github.com/huggingface/transformers/blob/a4fc0c80b11e14aaf6a9ec7c6fa5e6dab54261e4/src/transformers/generation_utils.py#L110). I was looking for it in the body of the method.<|||||>> It turned out there is a decorator before the [generate() method](https://github.com/huggingface/transformers/blob/a4fc0c80b11e14aaf6a9ec7c6fa5e6dab54261e4/src/transformers/generation_utils.py#L110). I was looking for it in the body of the method.

Did you solve this problem? I run into the same problem and need a solution. Thanks!