RuntimeError: The expanded size of the tensor (449) must match the existing size (2) at non-singleton dimension 2.  Target sizes: [4, 2, 449].  Tensor sizes: [1, 2] while using ALBERT
## ❓ Questions & Help

<!-- A clear and concise description of the question. -->
I wanted to use ALBERT with a double head 
as we have one  for openaigpt with the name OpenAIGPTDoubleHeadsModel
I am doing it with taking inspiration from OpenAIGPTDoubleHeadsModel
but I am getting this error

` File "train.py", line 266, in <module>
    train()
  File "train.py", line 258, in train
    trainer.run(train_loader, max_epochs=args.n_epochs)
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/ignite/engine/engine.py", line 446, in run
    self._handle_exception(e)
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/ignite/engine/engine.py", line 410, in _handle_exception
    raise e
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/ignite/engine/engine.py", line 433, in run
    hours, mins, secs = self._run_once_on_dataset()
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/ignite/engine/engine.py", line 399, in _run_once_on_dataset
    self._handle_exception(e)
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/ignite/engine/engine.py", line 410, in _handle_exception
    raise e
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/ignite/engine/engine.py", line 391, in _run_once_on_dataset
    self.state.output = self._process_function(self, batch)
  File "train.py", line 180, in update
    mc_labels=mc_labels, lm_labels=lm_labels
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/transformers/modeling_albert.py", line 956, in forward
    inputs_embeds=inputs_embeds)
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/transformers/modeling_albert.py", line 499, in forward
    inputs_embeds=inputs_embeds)
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/moha/venv/huggtrans/lib/python3.6/site-packages/transformers/modeling_bert.py", line 171, in forward
    position_ids = position_ids.unsqueeze(0).expand(input_shape)
RuntimeError: The expanded size of the tensor (449) must match the existing size (2) at non-singleton dimension 2.  Target sizes: [4, 2, 449].  Tensor sizes: [1, 2]`


the code block for AlbertDoubleHeadsModel 


`class AlbertDoubleHeadsModel(AlbertPreTrainedModel):`
`    def __init__(self, config):`
`        super(AlbertDoubleHeadsModel, self).__init__(config)`

        self.albert = AlbertModel(config)
        self.lm_head = nn.Linear(config.embedding_size, config.vocab_size, bias=False)
        self.multiple_choice_head = SequenceSummary(config)

        self.init_weights()

    def get_output_embeddings(self):
        return self.lm_head

    def forward(self, input_ids=None, attention_mask=None, token_type_ids=None, position_ids=None, head_mask=None, inputs_embeds=None,
                mc_token_ids=None, lm_labels=None, mc_labels=None):
        transformer_outputs = self.albert(input_ids,
                                               attention_mask=attention_mask,
                                               token_type_ids=token_type_ids,
                                               position_ids=position_ids,
                                               head_mask=head_mask,
                                               inputs_embeds=inputs_embeds)
        hidden_states = transformer_outputs[0]

        lm_logits = self.lm_head(hidden_states)
        mc_logits = self.multiple_choice_head(hidden_states, mc_token_ids).squeeze(-1)

        outputs = (lm_logits, mc_logits) + transformer_outputs[1:]
        if mc_labels is not None:
            loss_fct = CrossEntropyLoss()
            loss = loss_fct(mc_logits.view(-1, mc_logits.size(-1)),
                            mc_labels.view(-1))
            outputs = (loss,) + outputs
        if lm_labels is not None:
            shift_logits = lm_logits[..., :-1, :].contiguous()
            shift_labels = lm_labels[..., 1:].contiguous()
            loss_fct = CrossEntropyLoss(ignore_index=-1)
            loss = loss_fct(shift_logits.view(-1, shift_logits.size(-1)),
                            shift_labels.view(-1))
            outputs = (loss,) + outputs

        return outputs  # (lm loss), (mc loss), lm logits, mc logits, (all hidden_states), (attentions)`



Is there anything that I am missing?
Please do tell me.

What is the code that you are executing that leads to this error? <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>How did you fix the problem?