If you're interested in experimenting with a minimal version of the [`UNet2DConditionModel`] used in SDXL, take a look at the [minSDXL](https://github.com/cloneofsimo/minSDXL) implementation which is written in PyTorch and directly compatible with 🤗 Diffusers.
/Users/nielsrogge/Documents/python_projecten/diffusers/docs/source/en/using-diffusers/sdxl.md