Load the image encoder with [`~transformers.CLIPVisionModelWithProjection`].  
```py
import torch
from diffusers import AutoPipelineForText2Image, DDIMScheduler
from transformers import CLIPVisionModelWithProjection
from diffusers.utils import load_image