DFloat request — Qwen-Image

[kabachuha]

Qwen Image is a 20b model capable of awesome GPT-level image generation and editing. https://github.com/QwenLM/Qwen-Image

However, it's about 40gbs in size, which makes it quite impractical.

Having a 30% reduced DFloat11 request might fit it into around 29-30GBs, making it available for lossless inference on High-End consumer devices. (And even more, with offloading)

[tonyzhang617]

Done! https://huggingface.co/DFloat11/Qwen-Image-DF11

The compressed model now fits on a single 32GB GPU, or even a single 16GB GPU with offloading!

Please give it a try and let me know how it works for you.

If you find it helpful, I'd really appreciate it if you could help spread the word!

[blargg]

I tried this, but couldn't get it to run it successfully on 16GB.

For example:

python dfloat_example.py --cpu_offload

Will take up all of my ram and swap (64 GB total) and then freeze my computer.

python dfloat_example.py --cpu_offload --cpu_offload_block 10 --prompt "man on a bike"

Would crash because cuda could not allocate memory on the GPU.

What settings did you use on your 16GB run? I'll try those out.