Model2Model inference
## ❓ Questions & Help
I am trying to implement a simple Model2Model question-answer task, where the input is question and the answer needs to be generated. At inference time I feed in the "[CLS]" token to let it generate but it only generates a single token instead of the entire sentence. The ppl at which I do the inference is ~5 ppl on valid set. 

Is there a fundamental issue with my model?

Training time 
`outputs = model(input_ids,
                        batch['speakableAnswer'],
                        decoder_lm_labels=batch['speakableAnswer'])`
print(outputs.size() -> batch x seq_len x vocab_size)

Test time
`outputs = model(input_ids,
                        batch['speakableAnswer'])`
print(outputs.size() -> batch x 1 x vocab_size)
<!-- A clear and concise description of the question. -->

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
