title: Continuous Speculative Decoding for Autoregressive Image Generation
summary: Continuous-valued Autoregressive (AR) image generation models have
  demonstrated notable superiority over their discrete-token counterparts,
  showcasing considerable reconstruction quality and higher generation fidelity.
  However, the computational demands of the autoregressive framework result in
  significant inference overhead. While speculative decoding has proven effective
  in accelerating Large Language Models (LLMs), their adaptation to
  continuous-valued visual autoregressive models remains unexplored. This work
  generalizes the speculative decoding algorithm from discrete tokens to
  continuous space. By analyzing the intrinsic properties of output distribution,
  we establish a tailored acceptance criterion for the diffusion distributions
  prevalent in such models. To overcome the inconsistency that occurred in
  speculative decoding output distributions, we introduce denoising trajectory
  alignment and token pre-filling methods. Additionally, we identify the
  hard-to-sample distribution in the rejection phase. To mitigate this issue, we
  propose a meticulous acceptance-rejection sampling method with a proper upper
  bound, thereby circumventing complex integration. Experimental results show
  that our continuous speculative decoding achieves a remarkable 2.33times
  speed-up on off-the-shelf models while maintaining the output distribution.
  Codes will be available at https://github.com/MarkXCloud/CSpD
paper_authors._id: 673d4d484bec167076968b8c
paper_authors.name: Zili Wang
paper_authors.status: claimed_verified
paper_authors.statusLastChangedAt: 2024-11-20T10:08:39.427Z
paper_authors._id: 6410998db27543634e3aacc3
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/1678809388600-noauth.png
paper_authors.fullname: Zili Wang
paper_authors.type: user
paper_authors.user: MarkWang
paper_authors._id: 673d4d484bec167076968b8d
paper_authors.name: Robert Zhang
paper_authors._id: 673d4d484bec167076968b8e
paper_authors.name: Kun Ding
paper_authors._id: 673d4d484bec167076968b8f
paper_authors.name: Qi Yang
paper_authors._id: 673d4d484bec167076968b90
paper_authors.name: Fei Li
paper_authors._id: 673d4d484bec167076968b91
paper_authors.name: Shiming Xiang
paper_title: Continuous Speculative Decoding for Autoregressive Image Generation
paper_summary: Continuous-valued Autoregressive (AR) image generation models have
  demonstrated notable superiority over their discrete-token counterparts,
  showcasing considerable reconstruction quality and higher generation fidelity.
  However, the computational demands of the autoregressive framework result in
  significant inference overhead. While speculative decoding has proven effective
  in accelerating Large Language Models (LLMs), their adaptation to
  continuous-valued visual autoregressive models remains unexplored. This work
  generalizes the speculative decoding algorithm from discrete tokens to
  continuous space. By analyzing the intrinsic properties of output distribution,
  we establish a tailored acceptance criterion for the diffusion distributions
  prevalent in such models. To overcome the inconsistency that occurred in
  speculative decoding output distributions, we introduce denoising trajectory
  alignment and token pre-filling methods. Additionally, we identify the
  hard-to-sample distribution in the rejection phase. To mitigate this issue, we
  propose a meticulous acceptance-rejection sampling method with a proper upper
  bound, thereby circumventing complex integration. Experimental results show
  that our continuous speculative decoding achieves a remarkable 2.33times
  speed-up on off-the-shelf models while maintaining the output distribution.
  Codes will be available at https://github.com/MarkXCloud/CSpD
paper_ai_summary: A speculative decoding algorithm is adapted for continuous-valued image generation to reduce inference time while preserving output quality.
paper_ai_keywords: Continue-valued Autoregressive models speculative decoding Large Language Models diffusion distributions denoising trajectory alignment token pre-filling acceptance-rejection sampling