How to insert a hidden output from GPT2 model directly into a BERT layer?
Hello,

I am trying to do the following:
1. Feed in the input_ids into the embedding layer of the pre-trained GPT2 model, and get the resulting embedding.
2. directly feed the embedding from 1. to each layer of the pre-trained GPT2, by using `GPT2_model.transformers.h`.
Store the resulting hidden output from each layer of the GPT-2 in a tensor named `layer_hidden_state`.
3. Directly input the `layer_hidden_state` into the 1st layer (which is on the top of the embedding layer) of the pre-trained BertModel, and let BertModel process the `layer_hidden_state` until it reaches the uppermost layer of BertModel.

My attempts for carrying out the above steps are shown below.
But I am getting an error when I try to do the step 3...how can I fix my error? the error is shown at the bottom of my code.

Thank you for the help,

```Python
    # turn on the evaluation mode
    # (to prevent the dropout for evaluation purpose).
    gpt2DoubleHeadsModel.eval()
    len_input_ids = len(input_ids)

    
    # get the hidden state vector from the embedding layer.
    # we will use this hidden state vector as an input to each layer.
    input_hidden_state = gpt2DoubleHeadsModel(input_ids=input_ids,
                                                         mc_token_ids = mc_token_ids,
                                                         token_type_ids = token_type_ids,
                                                         attention_mask = attention_mask)[3][0][:,:,:].detach()

    for j in range(num_layer_gpt2):
            
        # directly feed in the embedding hidden state vector into each layer of the GPT2DoubleHeadsModel,
        # and retrieve the resulting hidden state vector from each layer.
        layer_hidden_state = \
                    gpt2DoubleHeadsModel.transformer.h[j](input_hidden_state)[0][:,(len_input_ids-1),:]
            
        # store the hidden state vectors of the last token from each layer in `last_hidden_output_tensor`.
        last_hidden_output_tensor[:,j,:] = layer_hidden_state
        
    last_hidden_output_tensor = tuple(last_hidden_output_tensor)
        
    best_model_bert = BertModel.from_pretrained('bert-large-uncased', output_hidden_states=True)
    
    # an error is generated here the error is shown below:    
    for k in range(nlayer_bert):
        last_hidden_output_tensor = best_model_bert.encoder.layer[k]((last_hidden_output_tensor)[0])

"""
 error: 
  File "/Users/hyunjindominiquecho/opt/anaconda3/lib/python3.7/site-packages/transformers/modeling_bert.py", line 239, in transpose_for_scores
    return x.permute(0, 2, 1, 3)

RuntimeError: number of dims don't match in permute
"""
```
how do you fix it ?