How to Paraphrase with GPT2?
# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

<!-- You should first ask your question on SO, and only if
     you didn't get an answer ask it here on GitHub. -->


You should rather use a seq2seq model for paraphrasing like T5 or BART. But if you want to do it using GPT-2 then maybe you can use this format

`input: input_text paraphrase: parahrase_text`

 while training,  set attention mask to 0 on the paraphrased text

and when generating just pass `input: input_text paraphrase: `  and sample till the `eos` token<|||||>Thank you. I'll be sure to try that! I know there are datasets online that I can use, but are there edits I should make to ensure that I get a suitable output? I've read something about cheating, and I want to avoid that. (I'm sorry if I am using the wrong terminology)<|||||>Can we use `T5` or `BART` like we would `GPT-2`?<|||||>hi @shamoons , yes you can, have a look at this https://madewithml.com/projects/1094/paraphrase-any-question-with-t5-text-to-text-transformer/<|||||>This looks like it’s only for questions. What about arbitrary sentences?<|||||>> This looks like it’s only for questions. What about arbitrary sentences?

You can do it with arbitrary sentences as well, but you'll need to fine-tune it yourself.<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>> hi @shamoons , yes you can, have a look at this https://madewithml.com/projects/1094/paraphrase-any-question-with-t5-text-to-text-transformer/
This url is gone.
<|||||>@kingglory 

This is the [url](https://github.com/ramsrigouthamg/Paraphrase-any-question-with-T5-Text-To-Text-Transfer-Transformer-) for the repo of that project.

Also there are quite a few paraphrase models on the [hub](https://huggingface.co/models?search=paraphrase) that you can try