What inference speeds do you guys get?

[kalebbroo]

I have recently been testing this and I cant seem to get speeds comparable to Comfy. I was just wondering if that was expected or if I was missing something. I created an image in Comfy and then SDcpp and it seems to be 1.5x-2x slower on SDcpp.

Is this normal? I have tried playing with all the cache and perf args but nothing really helps.

[stduhpf]

It depends on the backend you're using I guess. On my end, comparing ROCm backend to ComfyUI-Zluda, sd.cpp is about 2x faster.

[kalebbroo]

https://github.com/HartsyAI/SwarmUI-SD.cpp-Backend

I have been using CUDA. I assumed that would be the fastest. I made an extension to SwarmUI to use SDcpp and its working great. I just need to figure out that speed issue im having and I think I can start showing people how perfect this is to swap out in Swarm.

[dobkeratops]

I was seeing this on the Asus Ascent GX10 .. stablediffusion.cpp running z-image-turbo was ballpark half the rate of comfyUI. (which is a shame because I really like the convenience of a native inference engine). (let me double check that I really was comparing like for like with model quantisation etc)
quizzing LLMs about it i'm seeing suggestions that there might not be a 'fused im2col & GEMM' implementation here, but there might be WIP in forks about this? EDIT: actually i'm not even sure if that matters as much in a DiT model?? is that just used in the VAE ? might be something else entirely.

I had used stablediffusion.cpp with flux on an RTX4090 before this (where I didn't compare to anything else)