content.raw: 5 New implementations of Diffusion Models
content.type: text
content.value: 5 New implementations of Diffusion Models
content.type: new_line
content.type: new_line
content.raw: Diffusion models are widely used for image and video generation but remain underexplored in text generation, where autoregressive models (ARMs) dominate. Unlike ARMs, which produce tokens sequentially, diffusion models iteratively refine noise through denoising steps, offering greater flexibility and speed.
content.type: text
content.value: Diffusion models are widely used for image and video generation but remain underexplored in text generation, where autoregressive models (ARMs) dominate. Unlike ARMs, which produce tokens sequentially, diffusion models iteratively refine noise through denoising steps, offering greater flexibility and speed.
content.type: new_line
content.raw: Recent advancements show a shift toward using diffusion models in place of, or alongside, ARMs. Researchers also combine strengths from both methods and integrate autoregressive concepts into diffusion.
content.type: text
content.value: Recent advancements show a shift toward using diffusion models in place of, or alongside, ARMs. Researchers also combine strengths from both methods and integrate autoregressive concepts into diffusion.
content.type: new_line
content.type: new_line
content.raw: Here are 5 new implementations of diffusion models:
content.type: text
content.value: Here are 5 new implementations of diffusion models:
content.type: new_line
content.type: new_line
content.raw: 1. Mercury family of diffusion LLMs (dLLMs) by Inception Labs -> 
content.type: text
content.value: 1. Mercury family of diffusion LLMs (dLLMs) by Inception Labs -> 
content.href: https://www.inceptionlabs.ai/news
content.raw: https://www.inceptionlabs.ai/news
content.type: link
content.type: new_line
content.raw: It applies diffusion to text and code data, enabling sequence generation 10x faster than today's top LLMs. Now available Mercury Coder can run at over 1,000 tokens/sec on NVIDIA H100s.
content.type: text
content.value: It applies diffusion to text and code data, enabling sequence generation 10x faster than today's top LLMs. Now available Mercury Coder can run at over 1,000 tokens/sec on NVIDIA H100s.
content.type: new_line
content.type: new_line
content.raw: 2. Diffusion of Thoughts (DoT) -> 
content.type: text
content.value: 2. Diffusion of Thoughts (DoT) -> 
content.label: Diffusion of Thoughts: Chain-of-Thought Reasoning in Diffusion Language
    Models (2402.07754)
content.raw: https://huggingface.co/papers/2402.07754
content.id: 2402.07754
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2402.07754
content.type: new_line
content.raw: Integrates diffusion models with Chain-of-Thought. DoT allows reasoning steps to diffuse gradually over time. This flexibility enables balancing between reasoning quality and computational cost.
content.type: text
content.value: Integrates diffusion models with Chain-of-Thought. DoT allows reasoning steps to diffuse gradually over time. This flexibility enables balancing between reasoning quality and computational cost.
content.type: new_line
content.type: new_line
content.raw: 3. LLaDA -> 
content.type: text
content.value: 3. LLaDA -> 
content.label: Large Language Diffusion Models (2502.09992)
content.raw: https://huggingface.co/papers/2502.09992
content.id: 2502.09992
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2502.09992
content.type: new_line
content.raw: Shows diffusion models' potential in replacing ARMs. Trained with pre-training and SFT, LLaDA masks tokens, predicts them via a Transformer, and optimizes a likelihood bound. LLaDA matches key LLM skills, and surpasses GPT-4o in reversal poetry.
content.type: text
content.value: Shows diffusion models' potential in replacing ARMs. Trained with pre-training and SFT, LLaDA masks tokens, predicts them via a Transformer, and optimizes a likelihood bound. LLaDA matches key LLM skills, and surpasses GPT-4o in reversal poetry.
content.type: new_line
content.type: new_line
content.raw: 4. LanDiff -> 
content.type: text
content.value: 4. LanDiff -> 
content.label: The Best of Both Worlds: Integrating Language Models and Diffusion
    Models for Video Generation (2503.04606)
content.raw: https://huggingface.co/papers/2503.04606
content.id: 2503.04606
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2503.04606
content.type: new_line
content.raw: This hybrid text-to-video model combines autoregressive and diffusion paradigms, introducing a semantic tokenizer, an LM for token generation, and a streaming diffusion model. LanDiff outperforms models like Sora.
content.type: text
content.value: This hybrid text-to-video model combines autoregressive and diffusion paradigms, introducing a semantic tokenizer, an LM for token generation, and a streaming diffusion model. LanDiff outperforms models like Sora.
content.type: new_line
content.type: new_line
content.raw: 5. General Interpolating Discrete Diffusion (GIDD) -> 
content.type: text
content.value: 5. General Interpolating Discrete Diffusion (GIDD) -> 
content.label: Generalized Interpolating Discrete Diffusion (2503.04482)
content.raw: https://huggingface.co/papers/2503.04482
content.id: 2503.04482
content.type: paper
content.type: resource
content.url: https://huggingface.co/papers/2503.04482
content.type: new_line
content.raw: A flexible noising process with a novel diffusion ELBO enables combining masking and uniform noise, allowing diffusion models to correct mistakes, where ARMs struggle.
content.type: text
content.value: A flexible noising process with a novel diffusion ELBO enables combining masking and uniform noise, allowing diffusion models to correct mistakes, where ARMs struggle.
rawContent: 5 New implementations of Diffusion Models
  
  Diffusion models are widely used for image and video generation but remain underexplored in text generation, where autoregressive models (ARMs) dominate. Unlike ARMs, which produce tokens sequentially, diffusion models iteratively refine noise through denoising steps, offering greater flexibility and speed.
  Recent advancements show a shift toward using diffusion models in place of, or alongside, ARMs. Researchers also combine strengths from both methods and integrate autoregressive concepts into diffusion.
  
  Here are 5 new implementations of diffusion models:
  
  1. Mercury family of diffusion LLMs (dLLMs) by Inception Labs -> https://www.inceptionlabs.ai/news
  It applies diffusion to text and code data, enabling sequence generation 10x faster than today's top LLMs. Now available Mercury Coder can run at over 1,000 tokens/sec on NVIDIA H100s.
  
  2. Diffusion of Thoughts (DoT) -> https://huggingface.co/papers/2402.07754
  Integrates diffusion models with Chain-of-Thought. DoT allows reasoning steps to diffuse gradually over time. This flexibility enables balancing between reasoning quality and computational cost.
  
  3. LLaDA -> https://huggingface.co/papers/2502.09992
  Shows diffusion models' potential in replacing ARMs. Trained with pre-training and SFT, LLaDA masks tokens, predicts them via a Transformer, and optimizes a likelihood bound. LLaDA matches key LLM skills, and surpasses GPT-4o in reversal poetry.
  
  4. LanDiff -> https://huggingface.co/papers/2503.04606
  This hybrid text-to-video model combines autoregressive and diffusion paradigms, introducing a semantic tokenizer, an LM for token generation, and a streaming diffusion model. LanDiff outperforms models like Sora.
  
  5. General Interpolating Discrete Diffusion (GIDD) -> https://huggingface.co/papers/2503.04482
  A flexible noising process with a novel diffusion ELBO enables combining masking and uniform noise, allowing diffusion models to correct mistakes, where ARMs struggle.
reactions.reaction: 🚀
reactions.users: John6666 Norm BrigitteTousi huyzed chiungyit dantezxcd
reactions.reaction: 🧠
reactions.users: prithivMLmods Kseniase dantezxcd
reactions.reaction: 🔥
reactions.users: Kseniase JohnRoger dantezxcd
reactions.reaction: 👀
reactions.users: robb-0 dantezxcd gakadam
commentators._id: 64838b28c235ef76b63e4999
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/64838b28c235ef76b63e4999/ZhQCYoU3vps71Ag7Jezj6.jpeg
commentators.fullname: Ksenia Se
commentators.name: Kseniase
commentators.type: user
commentators._id: 6758a9850e3fff481964ca6d
commentators.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/no-auth/EolfJfjW25hC4Bt_hCPq8.png
commentators.fullname: Jean Louis
commentators.name: JLouisBiz
commentators.type: user