flymy-ai

/

qwen-image-realism-lora

like 69

Follow

FlyMy.AI 51

Hey! Thanks for your question.

We have added a ComfyUI workflow example. Our LoRAs are fully compatible with ComfyUI workflows, so please check your versions and make sure everything is set up according to the instructions:

https://github.com/FlyMyAI/flymyai-lora-trainer?tab=readme-ov-file#%EF%B8%8F-using-with-comfyui

I tried adding this to a couple standard ComfyUI LoRA Loader nodes but didnt' seem to work and threw some errors, the usual looking thing when the LoRA doesn't match the model e.g.:

lora key not loaded: transformer.transformer_blocks.9.attn.to_v.lora.down.weight
lora key not loaded: transformer.transformer_blocks.9.attn.to_v.lora.up.weight

I was using it with https://huggingface.co/QuantStack/Qwen-Image-Distill-GGUF?show_file_info=Qwen_Image_Distill-Q6_K.gguf just to make things more interesting hah.

I'll keep an eye out on r/comfyui and my usual YT channels like BenjiAI to see if support for Qwen-Image LoRAs gets added as a node or something.

Possible related issue: https://github.com/comfyanonymous/ComfyUI/issues/9203

I have this same problem. My comfyUI install and nodes are all up to date, but the lora doesn't make any difference and throws up errors in the console.
I'm using qwen_image_fp8_e4m3fn.safetensors. Perhaps it doesn't work with this particular model?

I have to set Lora strength to 3 with this lora before i start seeing major changes. Is this intended? Going to strength 4 and it just looks bad and garbled.

Maybe it's the new ComfyUI Update affecting this lora?

Strength 2 looks like a seed change. I don't really see any skin detail improvements though. I'm using the 8 step lightning lora.

Yeah I tested it with updated comfyui using a variety of models e.g. GGUF Qwen_Image_Distill-Q6_K.gguf, fp8 qwen_image_distill_full_fp8_e4m3fn.safetensors, and original qwen_image fp8 qwen_image_fp8_e4m3fn.safetensorswith both loras: flymy_realism.safetensors and flymy-ai-og-lora.safetensors.

I set the seeds to fixed and removed all randomness and just A/B compared with and without LoRAs but can see almost no difference which suggests it isn't quite working as it still prints out those errors in the terminal... so not sure what is up..

Workflow is not attached to images in the assets. The lora throws errors in console like:
lora key not loaded: transformer_blocks.9.txt_mlp.net.2.alpha
lora key not loaded: transformer_blocks.9.txt_mlp.net.2.lora_down.weight
lora key not loaded: transformer_blocks.9.txt_mlp.net.2.lora_up.weight

it must be something to do with only working with diffusers, not in Comfy.

Hello! Thanks for your question.

You can find workflow ComfyUI example at:
https://github.com/FlyMyAI/flymyai-lora-trainer/blob/main/qwen_image_lora_example.json

@KvLove

Just to double check, I used your exact workflow linked above. https://github.com/FlyMyAI/flymyai-lora-trainer/blob/main/qwen_image_lora_example.json#L314

However,

1. I did have to change the model from qwen_image_bf16.safetensors to the smaller but should be similar quality qwen_image_distill_full_fp8_e4m3fn.safetensors and also tried qwen_image_fp8_e4m3fn.safetensors. This is easier for most people to in under 24GB VRAM.
2. See on line 314 is is trying to load a LoRA called valentin_final.safetensors however your two repos only contain flymy_realism.safetensors and pytorch_lora_weights.safetensors (maybe you renamed this second one ??) otherwise I can't find the LoRA safetensors file which you suggest would work. Am I missing something, or where can I find valentin_final.safetensors which would work with ComfyUI?
3. I tried the workflow anyway with both of the available safetensors LoRAs and they both throw the errors like lora key not loaded: transformer.transformer_blocks.8.attn.to_q.lora.up.weight and the output looks pretty much the same and does not look like your screenshot even with what appears to be the same settings and seed:

This is with qwen_image_distill_full_fp8_e4m3fn.safetensors

This is with qwen_image_fp8_e4m3fn.safetensors

So either:

1. It does not actually work with ComfyUI.
2. Perhaps you didn't upload the working LoRA or I can't find it?
3. Some other confusion. I don't know what a "Valentin" is supposed to look like?

Thanks!

@pizzaboy29 @Primavera89 @aniaina @fimguy01
okay, the errors are gone now and the first Qwen-Image LoRA I managed to et working was the new lightx2v/Qwen-Image-Lightningfollowing the tip here: https://huggingface.co/lightx2v/Qwen-Image-Lightning/discussions/1#6898af417d5acff01f997a0d

update comfyui and chose the nightly version in the manager

Despite manually running git pull and git checkout main I guess it wasn't picking up the latest version from main as it switches the git repo on startup or something.

Anyway, now it seems like this Realism LoRA in this repo is also working and can be combined with the Lighting 8 step and no errors.

Gotta go test some more now, good luck y'all!

Hello! Yes — thanks for the tests. It’s really important that we keep the ComfyUI codebase up to date.