<p>One of my colleagues gave me a solution. (Thanks Ken!)</p>
<p><span class="math-container">$$
{\begin{aligned}
\sigma_{t|s}^2 &amp;= 1 - \alpha_{t|s}^2 \\
&amp;= 1 - {\alpha_t^2 \over \alpha_s^2} \\
&amp;= {\alpha_s^2 - \alpha_t^2 \over \alpha_s^2} \\
&amp;= {(1 - \sigma_s^2)-(1-\sigma_t^2) \over \alpha_s^2} \\
&amp;= {\sigma_t^2 - \sigma_s^2 \over \alpha_s^2} \\
&amp;= {\sigma_t^2 (\alpha_s^2 + \sigma_s^2) - \sigma_s^2 (\alpha_t^2 + \sigma_t^2) \over \alpha_s^2} \\
&amp;= {\sigma_t^2 \alpha_s^2 - \sigma_s^2 \alpha_t^2 \over \alpha_s^2} \\
&amp;= \left(1 -{\sigma_s^2 \alpha_t^2 \over \alpha_s^2 \sigma_t^2}\right) \sigma_t^2 \\
&amp;= \left(1 -{1 \over e^{\lambda_s}}\cdot e^{\lambda_t}\right) \sigma_t^2 \\
&amp;= \left(1 - e^{\lambda_t - \lambda_s}\right) \sigma_t^2 \\
\end{aligned}}
$$</span>
given
<span class="math-container">$$
\alpha_t^2 + \sigma_t^2 = 1 \\
\alpha_{t|s}^2 + \sigma_{t|s}^2 = 1 \\
e^{\lambda_t} = {\alpha_t^2 \over \sigma_t^2}
$$</span>
.</p>

CC BY-SA 4.0
2023-02-14T11:14:30.810Z
2023-02-14T11:14:30.810Z
hotohoto
<p>There is another derivation:
<span class="math-container">$$
{\begin{aligned}
\sigma_{t|s}^2 &amp;:= \sigma_t^2 - {\alpha_{t}^2 \over \alpha_s^2} \sigma_s^2 \\
&amp;= \left( 1 - {\alpha_t^2 \sigma_s^2\over \sigma_t^2\alpha_s^2} \right) \sigma_t^2\\
&amp;= \left(1 -e^{\lambda_t}\cdot {1 \over e^{\lambda_s}}\right) \sigma_t^2 \\
&amp;= \left(1 - e^{\lambda_t - \lambda_s}\right) \sigma_t^2 \\
\end{aligned}}
$$</span></p>
<p>given
<span class="math-container">$$
e^{\lambda_t} = {\alpha_t^2 \over \sigma_t^2}
$$</span>
.</p>

CC BY-SA 4.0
2023-02-15T12:33:37.067Z
2023-02-15T12:33:37.067Z
hotohoto
<p>Regarding Eq. (1) in <a href="https://arxiv.org/abs/2202.00512" rel="nofollow noreferrer">Progressive Distillation for Fast Sampling of Diffusion Models</a>,
<span class="math-container">$$
q(\mathbf{z}_t|\mathbf{z}_s) = \mathcal{N}(\mathbf{z}_t; (\alpha_t/\alpha_s)\mathbf{z}_s, \sigma _{t|s}^2 \mathbf{I})
\tag{1}
$$</span>
, it says <span class="math-container">$\sigma_{t|s}^2 = (1 - e^{\lambda_t - \lambda_s})\sigma_t^2$</span>.</p>
<p>I believe <span class="math-container">$\sigma_{t|s}$</span> is from DDPM's <span class="math-container">$\beta$</span>, then it's equivalent to <span class="math-container">$1 - \alpha_t^2 / \alpha_s^2$</span> here. Note that <span class="math-container">$\lambda_t = \log (\alpha_t^2 / \sigma_t^2)$</span>. As far as I understand, <span class="math-container">$\sigma_t^2 + \alpha_t^2 = 1$</span>. Maybe I'm taking something wrong.</p>
<p>Anyway, how could <span class="math-container">$\sigma_{t|s}^2 = (1 - e^{\lambda_t - \lambda_s})\sigma_t^2$</span> be derived from <span class="math-container">$1 - \alpha_t^2 / \alpha_s^2$</span>?</p>
<p>Some more notes.</p>
<ul>
<li>More papers are using the same expression.
<ul>
<li><a href="https://arxiv.org/abs/2207.12598" rel="nofollow noreferrer">Classifier-Free Diffusion Guidance</a></li>
<li><a href="https://arxiv.org/abs/2204.03458" rel="nofollow noreferrer">Video Diffusion Models</a></li>
<li><a href="https://arxiv.org/abs/2210.02303" rel="nofollow noreferrer">Imagen Video: High Definition Video Generation with Diffusion Models</a></li>
</ul>
</li>
<li>I think the definition of <span class="math-container">$\sigma_{t|s}$</span> Eq. (22) in <a href="https://arxiv.org/abs/2107.00630" rel="nofollow noreferrer">the original VDM paper</a> is straightforward with respect to DDPM's definitions.</li>
<li>The expression seems to be quite similar to Eq.(32) of VDM.</li>
</ul>

<diffusion-models>
How to derive the variance of the forward step of Variational Diffusion Models in terms of the log signal-to-noise ratio $\lambda_t$?
hotohoto
# How to derive the variance of the forward step of Variational Diffusion Models in terms of the log signal-to-noise ratio $\lambda_t$?

(Asked by: username_0 on 2023-02-14)

Regarding Eq. (1) in [Progressive Distillation for Fast Sampling of Diffusion Models](https://arxiv.org/abs/2202.00512),
$$
q(\mathbf{z}\_t|\mathbf{z}\_s) = \mathcal{N}(\mathbf{z}\_t; (\alpha\_t/\alpha\_s)\mathbf{z}\_s, \sigma \_{t|s}^2 \mathbf{I})
\tag{1}
$$
, it says $\sigma\_{t|s}^2 = (1 - e^{\lambda\_t - \lambda\_s})\sigma\_t^2$.

I believe $\sigma\_{t|s}$ is from DDPM's $\beta$, then it's equivalent to $1 - \alpha\_t^2 / \alpha\_s^2$ here. Note that $\lambda\_t = \log (\alpha\_t^2 / \sigma\_t^2)$. As far as I understand, $\sigma\_t^2 + \alpha\_t^2 = 1$. Maybe I'm taking something wrong.

Anyway, how could $\sigma\_{t|s}^2 = (1 - e^{\lambda\_t - \lambda\_s})\sigma\_t^2$ be derived from $1 - \alpha\_t^2 / \alpha\_s^2$?

Some more notes.

* More papers are using the same expression.
  + [Classifier-Free Diffusion Guidance](https://arxiv.org/abs/2207.12598)
  + [Video Diffusion Models](https://arxiv.org/abs/2204.03458)
  + [Imagen Video: High Definition Video Generation with Diffusion Models](https://arxiv.org/abs/2210.02303)
* I think the definition of $\sigma\_{t|s}$ Eq. (22) in [the original VDM paper](https://arxiv.org/abs/2107.00630) is straightforward with respect to DDPM's definitions.
* The expression seems to be quite similar to Eq.(32) of VDM.

**Tags:** diffusion-models


---

## 2 answers

### Answer by username_0 on 2023-02-14. Score: 1

One of my colleagues gave me a solution. (Thanks Ken!)

$$
{\begin{aligned}
\sigma\_{t|s}^2 &= 1 - \alpha\_{t|s}^2 \\
&= 1 - {\alpha\_t^2 \over \alpha\_s^2} \\
&= {\alpha\_s^2 - \alpha\_t^2 \over \alpha\_s^2} \\
&= {(1 - \sigma\_s^2)-(1-\sigma\_t^2) \over \alpha\_s^2} \\
&= {\sigma\_t^2 - \sigma\_s^2 \over \alpha\_s^2} \\
&= {\sigma\_t^2 (\alpha\_s^2 + \sigma\_s^2) - \sigma\_s^2 (\alpha\_t^2 + \sigma\_t^2) \over \alpha\_s^2} \\
&= {\sigma\_t^2 \alpha\_s^2 - \sigma\_s^2 \alpha\_t^2 \over \alpha\_s^2} \\
&= \left(1 -{\sigma\_s^2 \alpha\_t^2 \over \alpha\_s^2 \sigma\_t^2}\right) \sigma\_t^2 \\
&= \left(1 -{1 \over e^{\lambda\_s}}\cdot e^{\lambda\_t}\right) \sigma\_t^2 \\
&= \left(1 - e^{\lambda\_t - \lambda\_s}\right) \sigma\_t^2 \\
\end{aligned}}
$$
given
$$
\alpha\_t^2 + \sigma\_t^2 = 1 \\
\alpha\_{t|s}^2 + \sigma\_{t|s}^2 = 1 \\
e^{\lambda\_t} = {\alpha\_t^2 \over \sigma\_t^2}
$$
.


---

### Answer by username_0 on 2023-02-15. Score: 1

There is another derivation:
$$
{\begin{aligned}
\sigma\_{t|s}^2 &:= \sigma\_t^2 - {\alpha\_{t}^2 \over \alpha\_s^2} \sigma\_s^2 \\
&= \left( 1 - {\alpha\_t^2 \sigma\_s^2\over \sigma\_t^2\alpha\_s^2} \right) \sigma\_t^2\\
&= \left(1 -e^{\lambda\_t}\cdot {1 \over e^{\lambda\_s}}\right) \sigma\_t^2 \\
&= \left(1 - e^{\lambda\_t - \lambda\_s}\right) \sigma\_t^2 \\
\end{aligned}}
$$

given
$$
e^{\lambda\_t} = {\alpha\_t^2 \over \sigma\_t^2}
$$
.


---

