Skip to content

feat: add spectrum caching method#1322

Open
rmatif wants to merge 1 commit intoleejet:masterfrom
rmatif:spectrum
Open

feat: add spectrum caching method#1322
rmatif wants to merge 1 commit intoleejet:masterfrom
rmatif:spectrum

Conversation

@rmatif
Copy link
Contributor

@rmatif rmatif commented Mar 5, 2026

Yet another training-free acceleration method. This PR implements Adaptive Spectral Feature Forecasting for Diffusion Sampling Acceleration, currently for UNet only. For DiT models, we already have enough options in my view

This could replace and deprecate ucache, which was only ever an experimental method

Example usage:

/build/bin/sd-cli -m models/model.safetensors -p "a cute cat" --steps 20 -H 1024 -W 1024 --fa -s 42 --cache-mode spectrum --scheduler simple --sampling-method euler
Steps Baseline Spectrum
20 output_20_baseline output_20_spectrum
×1.0 ×1.82
30 output_30_baseline output_30_spectrum
×1.0 ×2.14
40 output_40_baseline output_40_spectrum
×1.0 ×2.50

@Green-Sky
Copy link
Contributor

For DiT models, we already have enough options in my view

Personally more interested in the DiT variant. They have some nice speed up and consistency claims.

Anyway, great work. :)

@GreenShadows
Copy link

Based on what is claimed in these papers, this method is likely to be more efficient in terms of speed and quality than current DITs.

@JohnLoveJoy
Copy link

Impressive how the quality is preserved in the 20-step example!

@wbruna
Copy link
Contributor

wbruna commented Mar 5, 2026

currently for UNet only. For DiT models, we already have enough options in my view

So... If I do request --cache-mode spectrum for Z-Image Turbo, and the log says it managed to skip 2 of the 9 steps, is it actually using another method?

[INFO ] stable-diffusion.cpp:3488 - generating image: 1/1 - seed 42
[INFO ] stable-diffusion.cpp:1812 - Spectrum enabled - w: 0.40, m: 3, lam: 1.00, window: 2, flex: 0.50, warmup: 4, stop: 90%
[DEBUG] ggml_extend.hpp:1772 - z_image compute buffer size: 548.22 MB(VRAM)
  |==================================================| 9/9 - 4.67s/it
[INFO ] stable-diffusion.cpp:2333 - Spectrum skipped 2/9 steps (1.29x estimated speedup)
[INFO ] stable-diffusion.cpp:3530 - sampling completed, taking 42.07s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants