---
library_name: transformers
tags: []
---

# Scaling Down Text Encoders of Text-to-Image Diffusion Models

Official Repository of the paper: *[Scaling Down Text Encoders of Text-to-Image Diffusion Models](https://github.com/LifuWang-66/DistillT5)*.

Project Page: https://github.com/LifuWang-66/DistillT5.git

## Model Descriptions:
T5-Base distilled from [T5-XXL](https://huggingface.co/google/flan-t5-xxl) using [Flux](https://huggingface.co/black-forest-labs/FLUX.1-dev). 
It is 50 times smaller and retains most capability of T5-XXL.

## Generation Results:

<p align="center">
    <img src="showcase.png">
</p>

## Usage:
1. Setup the environment:
```
git clone https://github.com/LifuWang-66/DistillT5.git
cd DistillT5
conda create -n distillt5 python=3.12
conda activate distillt5
pip install -r requirements.txt
pip install ./diffusers
```

2. Inference
```py
import sys
import os
sys.path.append(os.path.abspath(os.path.join(os.path.dirname(__file__), '..')))
from models.T5_encoder import T5EncoderWithProjection
import torch
from diffusers import FluxPipeline


pipe = FluxPipeline.from_pretrained("black-forest-labs/FLUX.1-dev", torch_dtype=torch.float16)
text_encoder = T5EncoderWithProjection.from_pretrained('LifuWang/DistillT5', torch_dtype=torch.float16)
pipe.text_encoder_2 = text_encoder
pipe = pipe.to('cuda')

prompt = "Photorealistic portrait of a stylish young woman wearing a futuristic golden sequined bodysuit that catches the light, creating a metallic, mirror-like effect. She is wearing large, reflective blue-tinted aviator sunglasses. Over her head, she wears headphones with metallic accents, giving a modern, cyber aesthetic."

image = pipe(prompt=prompt, num_images_per_prompt=1, guidance_scale=3.5, num_inference_steps=20).images[0]

image.save("t5_base.png")
```