How to feeding hidden state vectors from one transformer directly into a layer of different transformer
Hello,

I want to perform the following task:

**1.** _Feed in a text sequence to a GPT2Model (the GPT-2 model without output head);_

**2.** _Extract the hidden state vectors of the text sequence that are generated at each layer of the GPT2Model;_

**3.** _Feed the hidden state vectors from each layer of the GPT2Model as an input to the GPT2DoubleHeadsModel (assuming that the n_embd of the GPT2Model and the GPT2DoubleHeadsModel are equal), and calculate the **mc_loss** that results from inputting the hidden state vectors. My GPT2DoubleHeadsModel would only be consisted of (1 layer) + (the output heads)._


I know that the code below can be used to feed in hidden state vectors as inputs to each individual layer of a HuggingFace GPT-2 model:

```python
model.transformer.h[i](embed)
```
But the code above will only give me the output of an individual layer, so it wouldn't give me the mc_loss that I am looking for (since to calculating the mc_loss from the hidden state vectors, I would first need to feed in the hidden state vectors into the 1st layer of the GPT2DoubleHeadsModel, and somehow feed in the output of the layer as an input to the mc_head (multiple choice output head) of the GPT2DoubleHeadsModel) 

My questions are:
1. Is there any way to perform the task 3 with the Hugging Face GPT-2 models?
2. Is there anyway that I can access the GPT2DoubleHeadsModel's mc_head (multiple choice head), similar to the code that I wrote above?

Thank you,
Did you ever find the answer? 