title: Ψ-Sampler: Initial Particle Sampling for SMC-Based Inference-Time
    Reward Alignment in Score Models
summary: We introduce Psi-Sampler, an SMC-based framework incorporating pCNL-based
  initial particle sampling for effective inference-time reward alignment with a
  score-based generative model. Inference-time reward alignment with score-based
  generative models has recently gained significant traction, following a broader
  paradigm shift from pre-training to post-training optimization. At the core of
  this trend is the application of Sequential Monte Carlo (SMC) to the denoising
  process. However, existing methods typically initialize particles from the
  Gaussian prior, which inadequately captures reward-relevant regions and results
  in reduced sampling efficiency. We demonstrate that initializing from the
  reward-aware posterior significantly improves alignment performance. To enable
  posterior sampling in high-dimensional latent spaces, we introduce the
  preconditioned Crank-Nicolson Langevin (pCNL) algorithm, which combines
  dimension-robust proposals with gradient-informed dynamics. This approach
  enables efficient and scalable posterior sampling and consistently improves
  performance across various reward alignment tasks, including layout-to-image
  generation, quantity-aware generation, and aesthetic-preference generation, as
  demonstrated in our experiments.
paper_authors._id: 684124368cb0edba3ab8f738
paper_authors.name: Taehoon Yoon
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T12:43:45.956Z
paper_authors._id: 64df25a16466b911722ca1da
paper_authors.avatarUrl: /avatars/da92419e4d8ba412f163f85e09055f49.svg
paper_authors.fullname: TaeHoon Yoon
paper_authors.type: user
paper_authors.user: sillsill777
paper_authors._id: 684124368cb0edba3ab8f739
paper_authors.name: Yunhong Min
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T12:43:53.200Z
paper_authors._id: 66ee81b676a8038cb42c8caa
paper_authors.avatarUrl: /avatars/9b4c5ded9c94788c35ce7ffbc2f8d24b.svg
paper_authors.fullname: Yunhong Min
paper_authors.type: user
paper_authors.user: myhong
paper_authors._id: 684124368cb0edba3ab8f73a
paper_authors.name: Kyeongmin Yeo
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T12:44:00.132Z
paper_authors._id: 659e42cfb65ee9ee1fd11e61
paper_authors.avatarUrl: /avatars/a9220d099f32800fc43ae79bb519c1e9.svg
paper_authors.fullname: Kyeongmin Yeo
paper_authors.type: user
paper_authors.user: 32V
paper_authors._id: 684124368cb0edba3ab8f73b
paper_authors.name: Minhyuk Sung
paper_authors.status: admin_assigned
paper_authors.statusLastChangedAt: 2025-06-05T12:44:05.457Z
paper_authors._id: 631f432b5ba8c026340a7890
paper_authors.avatarUrl: https://cdn-avatars.huggingface.co/v1/production/uploads/631f432b5ba8c026340a7890/9PK7A_TRMpugwYjCsNBf1.jpeg
paper_authors.fullname: Minhyuk Sung
paper_authors.type: user
paper_authors.user: Minhyuk
paper_title: Ψ-Sampler: Initial Particle Sampling for SMC-Based Inference-Time
    Reward Alignment in Score Models
paper_summary: We introduce Psi-Sampler, an SMC-based framework incorporating pCNL-based
  initial particle sampling for effective inference-time reward alignment with a
  score-based generative model. Inference-time reward alignment with score-based
  generative models has recently gained significant traction, following a broader
  paradigm shift from pre-training to post-training optimization. At the core of
  this trend is the application of Sequential Monte Carlo (SMC) to the denoising
  process. However, existing methods typically initialize particles from the
  Gaussian prior, which inadequately captures reward-relevant regions and results
  in reduced sampling efficiency. We demonstrate that initializing from the
  reward-aware posterior significantly improves alignment performance. To enable
  posterior sampling in high-dimensional latent spaces, we introduce the
  preconditioned Crank-Nicolson Langevin (pCNL) algorithm, which combines
  dimension-robust proposals with gradient-informed dynamics. This approach
  enables efficient and scalable posterior sampling and consistently improves
  performance across various reward alignment tasks, including layout-to-image
  generation, quantity-aware generation, and aesthetic-preference generation, as
  demonstrated in our experiments.
paper_ai_summary: The framework $\Psi$-Sampler uses SMC with pCNL for efficient posterior sampling and reward alignment in score-based generative models, enhancing performance across various tasks.
paper_ai_keywords: SMC-based framework pCNL-based initial particle sampling inference-time reward alignment score-based generative model Sequential Monte Carlo denoising process Gaussian prior reward-aware posterior preconditioned Crank-Nicolson Langevin layout-to-image generation quantity-aware generation aesthetic-preference generation