Answers.Body: <p>Based on our conversation in the comment section, what Mucida wants is a reformulation of the input, e.g. if the input is:</p>
  <p><strong>&quot;A company director has a pecuniary duty&quot;</strong></p>
  <p>the output should be:</p>
  <p><strong>A company director has a pecuniary duty</strong> or <strong>&quot;A CEO has monetary responsabilities&quot;</strong>.</p>
  <p>By default, GPT2 returns what could be the next sentence in a longer paragraph, e.g.:</p>
  <p><strong>&quot;It's not to make money, but to serve customers&quot;</strong>.</p>
  <p>When you want large language models like GPT2 to give you a certain type of answers, what usually works well is to give it a few examples of what you want as an input.</p>
  <p>For example, you'd give as input a few pairs of reformulations:</p>
  <p><strong>&quot;What's a reformulation of &quot;A company director has a pecuniary duty&quot;? It's &quot;A company director has a pecuniary duty&quot;. What's a reformulation of &quot;Stackoverflow is a great place to ask questions&quot;? It's &quot;Stackoverflow is where you get the best answers to your questions&quot;. What's a reformulation of &quot;Elon musk is the richest person on earth?&quot; It's&quot;</strong></p>
  <p>Then, the ouput of GPT2 should complete what comes after <strong>&quot;It's&quot;</strong> in the same style.</p>
  <p>You should check what's the context length of the GPT2 model you're using is. Otherwise, if you feed an input that's longer than the context length, it's not going to be taken into account in its entirety by the model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-07T14:22:43.580Z
Answers.Text: I think I will have to create my own decoding algorithm because it didn't work. I tried with a pretrained model for translation and for paraphrase. It stills try to complete a sentence
Answers.UserDisplayName: Mucida
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-08T19:00:45.317Z
Answers.Text: Can you show me the output you got from the model? Usually, getting what you want out of a big language model is a matter of crafting the right prompt.
Answers.UserDisplayName: Raphael Lopez Kaufman
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-09T20:22:20.480Z
Answers.Text: If a start the sentece with "A company director has a pecuniary duty.", it completes with:   "A company director has a pecuniary duty."I'm not going to say that I'm going to be a good manager," he said" ... word by word each iteration. A print would be easier to show you hehe
Answers.UserDisplayName: Mucida
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-10T20:05:01.810Z
Answers.Text: Did you try what I suggested? Giving a few examples of the type of completion you want as an input to the model? It seems to me you just tried again with a single sentence as an input
Answers.UserDisplayName: Raphael Lopez Kaufman
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-12T13:48:39.387Z
Answers.Text: I did, but didn't work. Sometimes it only repeats the first sentence, or completes with a random one. But I tried with only 6 pairs of sentences as examples, I will  try with more
Answers.UserDisplayName: Mucida
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-12T18:35:04.907Z
Answers.Text: I would try to figure out what's the context length of the model you're using
Answers.UserDisplayName: Raphael Lopez Kaufman
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-12T20:33:07.893Z
Answers.Text: Is there any way we can get in touch?
Answers.UserDisplayName: Mucida
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-05-06T18:02:16.517Z
Answers.LastActivityDate: 2022-05-06T18:02:16.517Z
Answers.UserDisplayName: Raphael Lopez Kaufman
Body: <p>I am using GPT2LMHeadModel to change the way GPT2 choose the next word in a sentence. At this point, I have to give the initial part of the sentence and GTP2 starts to predict the better next word.</p>
  <p>I want GPT2 to read an entire sentence and then start a new one based on that (like it does with translation)</p>
  <p>Is there any kind of parameter that I need to set up in order to make GPT2 start a sentence from zero, not complete an initial one?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-05T18:43:04.427Z
Comments.Text: You should be able to do this by feeding a period at the end of the sentence you're feeding to GPT2. It will understand that it needs to start a new sentence after that (because the most probably token after a period is the start of a new sentence)
Comments.UserDisplayName: Raphael Lopez Kaufman
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-06T11:21:16.753Z
Comments.Text: It's still trying to "complete" the context, for example: I gave "A company director has a pecuniary duty."  and gtp2 come up with "A company director has a pecuniary duty. It's not to make money, but to serve customers". It didn't started a new one based on the first. I am using a paraphrase pretrained model.
Comments.UserDisplayName: Mucida
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-06T12:07:41.093Z
Comments.Text: What would you have wanted as a completion from the model after this first sentence? This will help understanding what you want to achieve.
Comments.UserDisplayName: Raphael Lopez Kaufman
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-05-06T15:27:16.037Z
Comments.Text: A paraphrase, like:
  "A company director has a pecuniary duty." .....
  "A CEO has monetary responsabilities"
  
  Something like that
Comments.UserDisplayName: Mucida
Tags: <natural-language-processing><machine-translation><natural-language-generation><gpt-2>
Title: BERT2: How to use GPT2LMHeadModel to start a sentence, not complete it
UserDisplayName: Mucida
ThreadText: # BERT2: How to use GPT2LMHeadModel to start a sentence, not complete it
  
  (Asked by: username_0 on 2022-05-05)
  
  I am using GPT2LMHeadModel to change the way GPT2 choose the next word in a sentence. At this point, I have to give the initial part of the sentence and GTP2 starts to predict the better next word.
  
  I want GPT2 to read an entire sentence and then start a new one based on that (like it does with translation)
  
  Is there any kind of parameter that I need to set up in order to make GPT2 start a sentence from zero, not complete an initial one?
  
  **Tags:** natural-language-processing, machine-translation, natural-language-generation, gpt-2
  > **Comments:**
  > **username_1**: You should be able to do this by feeding a period at the end of the sentence you're feeding to GPT2. It will understand that it needs to start a new sentence after that (because the most probably token after a period is the start of a new sentence)  
  > **username_0**: It's still trying to "complete" the context, for example: I gave "A company director has a pecuniary duty."  and gtp2 come up with "A company director has a pecuniary duty. It's not to make money, but to serve customers". It didn't started a new one based on the first. I am using a paraphrase pretrained model.  
  > **username_1**: What would you have wanted as a completion from the model after this first sentence? This will help understanding what you want to achieve.  
  > **username_0**: A paraphrase, like: "A company director has a pecuniary duty." ..... "A CEO has monetary responsabilities"  Something like that  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-05-06. Score: 1
  
  Based on our conversation in the comment section, what Mucida wants is a reformulation of the input, e.g. if the input is:
  
  **"A company director has a pecuniary duty"**
  
  the output should be:
  
  **A company director has a pecuniary duty** or **"A CEO has monetary responsabilities"**.
  
  By default, GPT2 returns what could be the next sentence in a longer paragraph, e.g.:
  
  **"It's not to make money, but to serve customers"**.
  
  When you want large language models like GPT2 to give you a certain type of answers, what usually works well is to give it a few examples of what you want as an input.
  
  For example, you'd give as input a few pairs of reformulations:
  
  **"What's a reformulation of "A company director has a pecuniary duty"? It's "A company director has a pecuniary duty". What's a reformulation of "Stackoverflow is a great place to ask questions"? It's "Stackoverflow is where you get the best answers to your questions". What's a reformulation of "Elon musk is the richest person on earth?" It's"**
  
  Then, the ouput of GPT2 should complete what comes after **"It's"** in the same style.
  
  You should check what's the context length of the GPT2 model you're using is. Otherwise, if you feed an input that's longer than the context length, it's not going to be taken into account in its entirety by the model.
  
  > **Comments:**
  > **username_0**: I think I will have to create my own decoding algorithm because it didn't work. I tried with a pretrained model for translation and for paraphrase. It stills try to complete a sentence  
  > **username_1**: Can you show me the output you got from the model? Usually, getting what you want out of a big language model is a matter of crafting the right prompt.  
  > **username_0**: If a start the sentece with "A company director has a pecuniary duty.", it completes with:   "A company director has a pecuniary duty."I'm not going to say that I'm going to be a good manager," he said" ... word by word each iteration. A print would be easier to show you hehe  
  > **username_1**: Did you try what I suggested? Giving a few examples of the type of completion you want as an input to the model? It seems to me you just tried again with a single sentence as an input  
  > **username_0**: I did, but didn't work. Sometimes it only repeats the first sentence, or completes with a random one. But I tried with only 6 pairs of sentences as examples, I will  try with more  
  > **username_1**: I would try to figure out what's the context length of the model you're using  
  > **username_0**: Is there any way we can get in touch?  
  
  ---
  
  