Thank you Kirill, for sharing the pointers. I agree with you that BART and PEGASUS are better for text summarization, over decoder only models. However, I was curious if some one had experimented with GPT2 variants for text generation. I found some sample implementations online, but no metrics on the performance evaluation on standard datasets. I feel it is also not straightforward to run inference (e.g., summaries) for text generation using GPT2. Some caveats such as penalizing long summaries, using special tokens to use a decoder only model for training & inference and not-immediately-obvious decoding strategies make inference tricky IMO. I ran a few tests and found the performance to be way below-par, contrary to some claims made in papers about the obviousness of achieving performance improvement on supervised tasks using GPT2 style models, esp., for text generation. Nevertheless, I will keep looking and update this thread if I find any relevant articles or if I find a robust way of doing summary generation using GPT2.