Answers.Body: <p>The <a href="https://arxiv.org/pdf/2204.07705.pdf" rel="nofollow noreferrer">Super-NaturalInstructions paper</a> tested including negative (i.e., incorrect examples) during testing and instruction-tuning, along with adding explanations for why an included few-shot example was correct or incorrect. It seems like it doesn't improve accuracy, at least in their configuration (see Table 4).</p>
  <p>Although, if you're using a model like ChatGPT, as @nbro mentioned, you could probably just prompt the model to not do something.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-07-16T04:32:39.917Z
Answers.LastActivityDate: 2023-07-16T04:32:39.917Z
Answers.UserDisplayName: Alexander Wan
Answers.Body: <p>Actually, my previous answer is a bit out of date. There's a recent paper that does what you're looking for: <a href="https://arxiv.org/abs/2307.03214" rel="nofollow noreferrer">&quot;PREADD: Prefix-Adaptive Decoding for Controlled Text Generation.&quot;</a></p>
  <p>During inference-time, they do two forward-passes. The first is for your normal generation, and the second is prefixed with a prompt that causes undesirable outputs.</p>
  <p>In your example, your normal forward pass could be: &quot;Improve the student's grammar.&quot; and the bad forward pass could be &quot;You are a terrible language teacher. Improve the student's grammar.&quot; You can then sample tokens that are unlikely under this second (bad) forward-pass, but likely in the first forward-pass. See Section 3 in the paper for the exact formulation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-09-03T05:53:30.717Z
Answers.LastActivityDate: 2023-09-03T05:53:30.717Z
Answers.UserDisplayName: Alexander Wan
Answers.Body: <blockquote>
  <p>Does Negative Prompting Exist?</p>
  </blockquote>
  <p>Yes, e.g. in some text-to-image generation models such as <a href="https://app.leonardo.ai/ai-generations" rel="nofollow noreferrer">https://app.leonardo.ai/ai-generations</a>:</p>
  <p><a href="https://i.sstatic.net/aEOEj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/aEOEj.png" alt="enter image description here" /></a></p>
  <p>One can run such negative prompts on one's computer, e.g. with <a href="https://github.com/AUTOMATIC1111/stable-diffusion-webui" rel="nofollow noreferrer">Automatic1111's Stable Diffusion WebUI</a> (<a href="https://github.com/EmpireMediaScience/A1111-Web-UI-Installer" rel="nofollow noreferrer">Windows 10/11 installer</a>):</p>
  <p><a href="https://i.sstatic.net/8LUnM.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/8LUnM.png" alt="enter image description here" /></a></p>
  <p>The paper {1} describes one example of how negative prompts can be implemented in LLMs.</p>
  <hr />
  <p>References:</p>
  <ul>
  <li>{1} <a href="https://arxiv.org/abs/2306.17806" rel="nofollow noreferrer">Stay on topic with Classifier-Free Guidance</a>. <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Sanchez%2C+G" rel="nofollow noreferrer">Guillaume Sanchez</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Fan%2C+H" rel="nofollow noreferrer">Honglu Fan</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Spangher%2C+A" rel="nofollow noreferrer">Alexander Spangher</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Levi%2C+E" rel="nofollow noreferrer">Elad Levi</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Ammanamanchi%2C+P+S" rel="nofollow noreferrer">Pawan Sasanka Ammanamanchi</a>, <a href="https://arxiv.org/search/cs?searchtype=author&amp;query=Biderman%2C+S" rel="nofollow noreferrer">Stella Biderman</a>. arXiv:2306.17806.</li>
  </ul>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-08-17T23:39:24.533Z
Answers.LastActivityDate: 2023-09-04T02:34:25.947Z
Answers.LastEditDate: 2023-09-04T02:34:25.947Z
Answers.UserDisplayName: Franck Dernoncourt
Body: <p>All the prompt engineering techniques I've seen seem to focus on telling the model what to do e.g. Few-Shot Prompting.</p>
  <p>Is there any value in giving the model examples of what not to do? Can you link me to any papers/techniques on the topic?</p>
  <p>Example</p>
  <p>I am building a bot to improve students' foreign language writing skills.</p>
  <p>Bad output: Corrected spelling of 'heisse' to 'heiße' because 'heiße' is the correct spelling in German.</p>
  <p>Better output: Corrected spelling of 'heisse' to 'heiße' because 'ss' can be combined to form 'ß' in German.
  I could solve this specific problem using few-shot prompting. But really, I want to tell the model &quot;don't give answers like 'this is how it is done in German', instead explain what is being done and the reasons for it&quot;.</p>
  <p>I may have answered my own question there... just put it what I said above in the system prompt?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-13T00:29:59.147Z
Comments.Text: I think the question in the title is a bit vague and not very interesting. You can certainly create prompts where you ask the model not to do something. The more interesting question is - how effective are they and is there any research on the topic? So, I'd reformulate the title to be more specific. From my experience, if we ask the model not to be verbose, it can still be verbose. But this doesn't say much because if you ask the model to do something it might also not do it.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-13T03:40:40.330Z
Comments.Text: You can fine-tune davinci-003 for your case. With enough examples, it should start outputting the required information.
Comments.UserDisplayName: Chinmay
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-13T11:23:19.283Z
Comments.Text: @nbro yes the idea of them being effective/whether research exists was the idea
Comments.UserDisplayName: codeananda
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2023-07-13T11:23:39.507Z
Comments.Text: @Chinmay so you're saying that yes you can do negative prompting and that it is effective?
Comments.UserDisplayName: codeananda
Tags: <prompt><prompt-design>
Title: Does Negative Prompting Exist?
UserDisplayName: codeananda
ThreadText: # Does Negative Prompting Exist?
  
  (Asked by: username_0 on 2023-07-12)
  
  All the prompt engineering techniques I've seen seem to focus on telling the model what to do e.g. Few-Shot Prompting.
  
  Is there any value in giving the model examples of what not to do? Can you link me to any papers/techniques on the topic?
  
  Example
  
  I am building a bot to improve students' foreign language writing skills.
  
  Bad output: Corrected spelling of 'heisse' to 'heiße' because 'heiße' is the correct spelling in German.
  
  Better output: Corrected spelling of 'heisse' to 'heiße' because 'ss' can be combined to form 'ß' in German.
  I could solve this specific problem using few-shot prompting. But really, I want to tell the model "don't give answers like 'this is how it is done in German', instead explain what is being done and the reasons for it".
  
  I may have answered my own question there... just put it what I said above in the system prompt?
  
  **Tags:** prompt, prompt-design
  > **Comments:**
  > **username_1**: I think the question in the title is a bit vague and not very interesting. You can certainly create prompts where you ask the model not to do something. The more interesting question is - how effective are they and is there any research on the topic? So, I'd reformulate the title to be more specific. From my experience, if we ask the model not to be verbose, it can still be verbose. But this doesn't say much because if you ask the model to do something it might also not do it.  
  > **username_2**: You can fine-tune davinci-003 for your case. With enough examples, it should start outputting the required information.  
  > **username_0**: @nbro yes the idea of them being effective/whether research exists was the idea  
  > **username_0**: @Chinmay so you're saying that yes you can do negative prompting and that it is effective?  
  
  ---
  
  ## 3 answers
  
  ### Answer by username_3 on 2023-07-16. Score: 3
  
  The [Super-NaturalInstructions paper](https://arxiv.org/pdf/2204.07705.pdf) tested including negative (i.e., incorrect examples) during testing and instruction-tuning, along with adding explanations for why an included few-shot example was correct or incorrect. It seems like it doesn't improve accuracy, at least in their configuration (see Table 4).
  
  Although, if you're using a model like ChatGPT, as @nbro mentioned, you could probably just prompt the model to not do something.
  
  
  ---
  
  ### Answer by username_3 on 2023-09-03. Score: 3
  
  Actually, my previous answer is a bit out of date. There's a recent paper that does what you're looking for: ["PREADD: Prefix-Adaptive Decoding for Controlled Text Generation."](https://arxiv.org/abs/2307.03214)
  
  During inference-time, they do two forward-passes. The first is for your normal generation, and the second is prefixed with a prompt that causes undesirable outputs.
  
  In your example, your normal forward pass could be: "Improve the student's grammar." and the bad forward pass could be "You are a terrible language teacher. Improve the student's grammar." You can then sample tokens that are unlikely under this second (bad) forward-pass, but likely in the first forward-pass. See Section 3 in the paper for the exact formulation.
  
  
  ---
  
  ### Answer by username_4 on 2023-08-17. Score: 2
  
  > Does Negative Prompting Exist?
  
  Yes, e.g. in some text-to-image generation models such as <https://app.leonardo.ai/ai-generations>:
  
  [![enter image description here](https://i.sstatic.net/aEOEj.png)](https://i.sstatic.net/aEOEj.png)
  
  One can run such negative prompts on one's computer, e.g. with [Automatic1111's Stable Diffusion WebUI](https://github.com/AUTOMATIC1111/stable-diffusion-webui) ([Windows 10/11 installer](https://github.com/EmpireMediaScience/A1111-Web-UI-Installer)):
  
  [![enter image description here](https://i.sstatic.net/8LUnM.png)](https://i.sstatic.net/8LUnM.png)
  
  The paper {1} describes one example of how negative prompts can be implemented in LLMs.
  
  ---
  
  References:
  
  * {1} [Stay on topic with Classifier-Free Guidance](https://arxiv.org/abs/2306.17806). [Guillaume Sanchez](https://arxiv.org/search/cs?searchtype=author&query=Sanchez%2C+G), [Honglu Fan](https://arxiv.org/search/cs?searchtype=author&query=Fan%2C+H), [Alexander Spangher](https://arxiv.org/search/cs?searchtype=author&query=Spangher%2C+A), [Elad Levi](https://arxiv.org/search/cs?searchtype=author&query=Levi%2C+E), [Pawan Sasanka Ammanamanchi](https://arxiv.org/search/cs?searchtype=author&query=Ammanamanchi%2C+P+S), [Stella Biderman](https://arxiv.org/search/cs?searchtype=author&query=Biderman%2C+S). arXiv:2306.17806.
  
  
  ---
  
  