Body: <p>I have a dataset of 100000 documents each labelled with a topic to it. I want to create a model such that, given a topic, the model can generate a document from it. </p>
  
  <p>I came across language models <a href="https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf" rel="nofollow noreferrer">GPT</a>, GPT-2 and <a href="https://arxiv.org/pdf/1810.04805.pdf" rel="nofollow noreferrer">BERT</a>. I learned that they can be used for generation purposes. But I did not find anywhere whether they can generate sentences given only a word.</p>
  
  <p>I am inclined to use GPT for my task, but I am not sure how to proceed with it. I wanted to know whether it is possible or not? It would be helpful if anyone can help me give a start in the right direction.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-04T07:59:50.387Z
Comments.Text: Why rely on models like GPT and do it manually from the array?
Comments.UserDisplayName: abunickabhi
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-04-04T08:30:26.707Z
Comments.Text: Because I want to leverage the linguistics knowledge of the pre-trained models and just fine tune it to my specific task.
Comments.UserDisplayName: mayank agrawal
Tags: <natural-language-processing><bert><language-model><gpt>
Title: How can I generate a document from a single word using GPT or BERT?
UserDisplayName: mayank agrawal
ThreadText: # How can I generate a document from a single word using GPT or BERT?
  
  (Asked by: username_0 on 2019-04-03)
  
  I have a dataset of 100000 documents each labelled with a topic to it. I want to create a model such that, given a topic, the model can generate a document from it.
  
  I came across language models [GPT](https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf), GPT-2 and [BERT](https://arxiv.org/pdf/1810.04805.pdf). I learned that they can be used for generation purposes. But I did not find anywhere whether they can generate sentences given only a word.
  
  I am inclined to use GPT for my task, but I am not sure how to proceed with it. I wanted to know whether it is possible or not? It would be helpful if anyone can help me give a start in the right direction.
  
  **Tags:** natural-language-processing, bert, language-model, gpt
  
  > **Comments:**
  > **username_1**: Why rely on models like GPT and do it manually from the array?  
  > **username_0**: Because I want to leverage the linguistics knowledge of the pre-trained models and just fine tune it to my specific task.  
  
  ---
  
  ## 0 answers
  
  