Body: <p>I'm working on understanding why large language models (LLMs) sometimes produce repetitive outputs when generating text. For example, in tasks involving explanations or lists, the model might repeat the same phrase or structure multiple times.</p>
  <p>Here’s an example I encountered (simplified):
  &quot;In few-shot learning, we can use it for text classification, and few-shot learning can also be used for text summarization, and few-shot learning can be applied to...&quot;</p>
  <p><a href="https://i.sstatic.net/iVzsqLlj.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/iVzsqLlj.png" alt="The image shows a screenshot of a text generated by a large language model (LLM) with repeated occurrences of the phrase &quot;and few-shot learning is used for tasks such as&quot; throughout multiple sentences. The repetition highlights an issue where the model generates redundant text structures instead of diverse or concise explanations." /></a>
  <strong>My Observations and Questions:</strong></p>
  <ol>
  <li>What factors in the decoding process (e.g., beam search, sampling,
  or temperature) contribute to this behavior?</li>
  <li>Could biases in the training data cause this repetition, or is it primarily a result of the model's context-aware prediction mechanism?</li>
  <li>Are there specific techniques or decoding adjustments (e.g., repetition penalties, modifying temperature) that have worked well to reduce this issue?</li>
  </ol>
  <p><strong>What I’ve Tried:</strong></p>
  <ol>
  <li>Experimented with increasing temperature and top-k sampling but
  still saw some improvement only in certain contexts.</li>
  <li>Adding more diverse prompts, but the issue persists in outputs with similar
  sentence structures.</li>
  </ol>
  <p>I’d appreciate insights, suggestions, or links to relevant research or best practices!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-11-25T02:49:43.310Z
Comments.Text: These are all possible factors for this behavior. This doesn't sound like a production LLM, is it your local Llama3.1-70B-Instruct LLM?
Comments.UserDisplayName: cinch
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-11-27T01:31:11.763Z
Comments.Text: No, it's not a local Llama3.1-70B-Instruct LLM. It's the production-ready version hosted on Hugging Face Chat, so it is indeed a live and operational model.
Comments.UserDisplayName: Amdjed Bens
Tags: <natural-language-processing><large-language-models><text-generation>
Title: Why Do LLMs Generate Repetitive Outputs During Text Generation?
UserDisplayName: Amdjed Bens
ThreadText: # Why Do LLMs Generate Repetitive Outputs During Text Generation?
  
  (Asked by: username_0 on 2024-11-24)
  
  I'm working on understanding why large language models (LLMs) sometimes produce repetitive outputs when generating text. For example, in tasks involving explanations or lists, the model might repeat the same phrase or structure multiple times.
  
  Here’s an example I encountered (simplified):
  "In few-shot learning, we can use it for text classification, and few-shot learning can also be used for text summarization, and few-shot learning can be applied to..."
  
  [![The image shows a screenshot of a text generated by a large language model (LLM) with repeated occurrences of the phrase ](https://i.sstatic.net/iVzsqLlj.png)](https://i.sstatic.net/iVzsqLlj.png)
  **My Observations and Questions:**
  
  1. What factors in the decoding process (e.g., beam search, sampling,
     or temperature) contribute to this behavior?
  2. Could biases in the training data cause this repetition, or is it primarily a result of the model's context-aware prediction mechanism?
  3. Are there specific techniques or decoding adjustments (e.g., repetition penalties, modifying temperature) that have worked well to reduce this issue?
  
  **What I’ve Tried:**
  
  1. Experimented with increasing temperature and top-k sampling but
     still saw some improvement only in certain contexts.
  2. Adding more diverse prompts, but the issue persists in outputs with similar
     sentence structures.
  
  I’d appreciate insights, suggestions, or links to relevant research or best practices!
  
  **Tags:** natural-language-processing, large-language-models, text-generation
  > **Comments:**
  > **username_1**: These are all possible factors for this behavior. This doesn't sound like a production LLM, is it your local Llama3.1-70B-Instruct LLM?  
  > **username_0**: No, it's not a local Llama3.1-70B-Instruct LLM. It's the production-ready version hosted on Hugging Face Chat, so it is indeed a live and operational model.  
  
  ---
  
  ## 0 answers
  
  