xinsir

/

controlnet-union-sdxl-1.0

like 1.2k

for flux-dev but still:
https://github.com/huggingface/diffusers/issues/9081

also flux-dev, but still:
https://huggingface.co/XLabs-AI/flux-controlnet-canny

Yes, I am back. I am looking at it.

Welcome back! :)

A quick head up:
InstantX released a canny controlnet for Flux-dev and a union controlnet for Flux-dev.
XLabs-AI released a controlnet collection (v1 + v2) for Flux-dev and multiple standalone v3 controlnets like depth v3. The also released a fp8 version of flux-dev (last time I checked they required their own custom nodes and samplers(!)). They also provide a flux controlnet training script.
Lllyasviel released a nf4 version of flux-dev and fp8 version of flux-dev. Read this for explanation and differences (especially in VRAM usage).
City96 released multiple GGUF quantization of flux-dev and GGUF quantization of flux-schnell (last time I checked they required their own custom nodes, may be supported by Comfy itself already).
I don't know of any flux-schnell quants, at least non which are popular.

if you are looking for an USP:

• provide a version for flux-schnell which is more open and can be used commercially. such a version would enable generation services to provide controlnets.
• provide a version which support quantified flux models (nf4, gguf). I wasn't able to run the InstantX union controlnet on 16GB VRAM with a 1024x1024 image for example ( https://huggingface.co/InstantX/FLUX.1-dev-Controlnet-Union/discussions/8 ). such a version would enable more people to use controlnets.
• provide a promax union controlnet for flux-dev in Xinsir-quality people know and love. such a version would allow a fair quality comparison between InstantX, XLabs-AI and Xinsir.
• collaborate with InstantX and XLabs-AI

Shakker-Labs collaborated with X-Labs to create FLUX.1-dev-ControlNet-Union-Pro

Hoping you are able to train a model for Flux @xinsir - Your SDXL models are exceptional!!

Hoping you are able to train a model for Flux @xinsir - Your SDXL models are exceptional!! +1

controlnet training support in diffusers: https://github.com/huggingface/diffusers/pull/9324

JasperAI released some controlnet models for Flux https://huggingface.co/collections/jasperai/flux1-dev-controlnets-66f27f9459d760dcafa32e08

TheMisto.ai Anyline Controlnet https://huggingface.co/TheMistoAI/MistoLine_Flux.dev
AlimamaCreative FLUX.1-dev Controlnet Inpainting (Beta) https://huggingface.co/alimama-creative/FLUX.1-dev-Controlnet-Inpainting-Beta

And any plans for FLUX.1-schnell in particular? The license for FLUX.1-dev is not permissive enough for many organizations.

I upvote this, I would love to port our controlnet Union SDXL workflows to Flux Shnell for our product !