Hugging Face's logo

Diffusers documentation

Normalization layers

You are viewing v0.32.0 version. A newer version v0.32.1 is available.
Hugging Face's logo
Join the Hugging Face community

and get access to the augmented documentation experience

to get started

Normalization layers

Customized normalization layers for supporting various models in 🤗 Diffusers.

AdaLayerNorm

class diffusers.models.normalization.AdaLayerNorm

< >

( embedding_dim: int num_embeddings: typing.Optional[int] = None output_dim: typing.Optional[int] = None norm_elementwise_affine: bool = False norm_eps: float = 1e-05 chunk_dim: int = 0 )

Parameters

  • embedding_dim (int) — The size of each embedding vector.
  • num_embeddings (int, optional) — The size of the embeddings dictionary.
  • output_dim (int, optional) —
  • norm_elementwise_affine (bool, defaults to `False) —
  • norm_eps (bool, defaults to False) —
  • chunk_dim (int, defaults to 0) —

Norm layer modified to incorporate timestep embeddings.

AdaLayerNormZero

class diffusers.models.normalization.AdaLayerNormZero

< >

( embedding_dim: int num_embeddings: typing.Optional[int] = None norm_type = 'layer_norm' bias = True )

Parameters

  • embedding_dim (int) — The size of each embedding vector.
  • num_embeddings (int) — The size of the embeddings dictionary.

Norm layer adaptive layer norm zero (adaLN-Zero).

AdaLayerNormSingle

class diffusers.models.normalization.AdaLayerNormSingle

< >

( embedding_dim: int use_additional_conditions: bool = False )

Parameters

  • embedding_dim (int) — The size of each embedding vector.
  • use_additional_conditions (bool) — To use additional conditions for normalization or not.

Norm layer adaptive layer norm single (adaLN-single).

As proposed in PixArt-Alpha (see: https://arxiv.org/abs/2310.00426; Section 2.3).

AdaGroupNorm

class diffusers.models.normalization.AdaGroupNorm

< >

( embedding_dim: int out_dim: int num_groups: int act_fn: typing.Optional[str] = None eps: float = 1e-05 )

Parameters

  • embedding_dim (int) — The size of each embedding vector.
  • num_embeddings (int) — The size of the embeddings dictionary.
  • num_groups (int) — The number of groups to separate the channels into.
  • act_fn (str, optional, defaults to None) — The activation function to use.
  • eps (float, optional, defaults to 1e-5) — The epsilon value to use for numerical stability.

GroupNorm layer modified to incorporate timestep embeddings.

< > Update on GitHub

←Custom activation functions Utilities→