text generation like lorem ipsum but human readable
Hi guys,

Hope you are all well !

We would like to add a fake text generator based on transformers/gpt-2 for the wordpress module called https://github.com/bordoni/fakerpress. For now, It uses only an old and not comprehensible lorem ipsum generator.

Is it possible to generate fake text (paragraphs, headings, taxonomies) with transformers/gpt-2 based on type of topics like reddit or shakespare datasets.

Why it would be useful ? for creating fake wordpress with human readable content and also indexable by a full-text search engine module (eg manticore or elasticsearch).

Thanks in advance for any insights or inputs on that topic

Cheers,
X
For conditional output generation, you might want to take a look at `CTRL` or you would have to fine-tuned gpt2. To just produce "any" fake text, you could  just use `GPT2` out of the box as in this demo: 
https://transformer.huggingface.co/doc/gpt2-large<|||||>Thanks for your reply :-)

What is cTRL ? do you have any references ?<|||||>Sorry I should have linked that: 
https://huggingface.co/transformers/model_doc/ctrl.html

CTRL is a very big model so quite difficult to run on a local machine - it might be easier to fine-tuned gpt2. I think @mariamabarham knows well how to fine-tune gpt2 - maybe you can add a couple lines? :-) <|||||>I think You can use gt2 with [run_language_modeling.py](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py). You can consider the [TextDataset](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py#L66) class or [LineByLineDataset](https://github.com/huggingface/transformers/blob/master/examples/run_language_modeling.py#L107) or define your own dataset class that suits better with your data structure.<|||||>Do you have a code example ? I am a little bit a newbie in NLP ^^ <|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
<|||||>Found this https://colab.research.google.com/drive/1VI3oBIOQYsym2x5oOux7DTNhpdR0r4uw<|||||>This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
