Perturb Hidden-State in Encoder-Decoder Models
Hi All,

I'm fairly new at using huggingface - so I apologize if this is answered in the documentation. I've looked around and don't think this has been asked before. I'm trying to perturb and use the hidden-state of an encoder-decoder model on the summarization task. 

More specifically, I'd like to 

1. Get a fixed-length hidden state for a given input after passing it to an encoder model, 
2. perturb it, and 
3. pass it to the decoder model to get an output. 

1 and 2 seem straightforward. To get the hidden state I use:
` hidden_states = model.base_model.encoder(inputs).last_hidden_state`
and to get a fixed length embedding I'm getting the last element of this list as per this [discussion](https://github.com/huggingface/transformers/issues/1950). Perturbing this would be as simple as adding noise to a tensor.

As for the third part, I'm having difficulty using this perturbed hidden-state in the decoder to generate an output. I've looked through the code and it seems a lot of the steps are abstracted out to accommodate many kinds of models. 

From what I understand, we can modify the model.generate() method so that the perturbation is added. However, this doesn't necessarily work for me since I wanted to use this hidden-state for other purposes before passing it to the decoder.

The other approach would be to create a separate function that takes as input the hidden-state and uses the second part of the code from model.generate() to produce outputs.

Before I proceed to implement this, I was wondering if there is a simpler way using existing code to do this. 





Hi @vin-nag 

If you want to directly pass `hidden_states` then you could do it this way

```python
model = EncoderDecoderModel.from_pretrained("google/roberta2roberta_L-24_gigaword")
tok = AutoTokenizer.from_pretrained("google/roberta2roberta_L-24_gigaword")

article = """australian shares closed down #.# percent monday
following a weak lead from the united states and
lower commodity prices , dealers said ."""

enc = tok(article, return_tensors="pt")
hidden_states = model.encoder(**enc, return_dict=True)

# perturb the last_hidden_state
hidden_states.last_hidden_state = perturb(hidden_states.last_hidden_state)

gen_ids = model.generate(input_ids=None, encoder_outputs=hidden_states, attention_mask=enc["attention_mask"])
tok.batch_decode(gen_ids)
```<|||||>@patil-suraj Thank you so much!<|||||>Hi @patil-suraj, thanks for the script. How can I access the `loss` after perturbation?