How can I tweak the `Longformer` code to control the input of a `Longformer`'s layer?
Hello,
I have asked a similar question on the HuggingFace forum, but I didn't get a clear answer that I was hoping for.
I tried the following to control the input of a `Longformer` layer:
`best_model_longformer.longformer.encoder.layer[layer_index](my_input_hidden_vector)` 

which of course, does not work.


How can I tweak the `Longformer` (or `BERT`) code to control the input of a `Longformer`'s layer?
Would this do the trick?:
```Python

  self_attention_outputs = Longformer_model.LongformerLayer.forward(hidden_states,
        attention_mask=None,
        output_attentions=False)
 Longformer_model.LongformerLayer.ff_chunk(self, self_attention_outputs)
```
But in this case, I don't know how to properly access the `LongformerLayer` class. I don't know if `Longformer_model.LongformerLayer` is the proper way.

I am not a programmer and I really do need more help on this. I know this can be a bit cumbersome to answer, but could you please help me on this? Your help is much appreciated.

Thank you,
The answer you got on the forum is pretty much the only one we have: copy paste the model code and customize it to your needs. If you're not a programmer, then you will need to learn a bit of Python/PyTorch to do this, but apart from making sure each model file contains the full code of each model, there is little more we can do to help you. There is an API to use the models in the common cases, users are expected to make the customizations they want if that does not suit their needs.<|||||>Hello, thank you for your reply.
Sorry if my question sounds a bit silly, but I just have a question:

When I do `my_Longformer_model.encoder.layer[layer_index].forward(hidden_output,my_attention_mask,output_attention=False)`, this error is generated:
```python

  File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_longformer.py", line 384, in _sliding_chunks_query_key_matmul
    batch_size, seq_len, num_heads, head_dim = query.size()

ValueError: too many values to unpack (expected 4)
```

Here, `my_attention_mask` is the same attention mask that I would specify under the regular
```python
longformer_output= my_longformer_model(input_ids=input_ids,....,attention_mask=my_attention_mask)
```

why exactly is the above error generated, and how can I remedy it?


Thank you,