Easy Diffusion:

Cut down the startup time of Easy Diffusion by 4 seconds on warm start, and atleast 10 seconds on cold start.

tl;dr - Right now, what annoys me the most with Easy Diffusion is the startup time and friction, and how Easy Diffusion isn’t actually “easy” anymore - the default model (SD 1.5) is bad, users take too long to get productive, and new models require a lot…

Added support for Krea2 in Easy Diffusion v4, in the beta channel.

Added support for Krea2 in Easy Diffusion v4, in the beta channel.

Upgraded stable-diffusion.cpp in Easy Diffusion (and sdkit) to the latest version.

Just completed another round of fixing support issues in Easy Diffusion.

Released Easy Diffusion v4.3 (which updates to sdkit v3.2).

Development update for Easy Diffusion: the beta branch has been merged into main, so this releases v3.5 (webui) and v4 to everyone.

Got Easy Diffusion v4 working on Apple and Intel Macs.

Started the long-pending rewrite of Easy Diffusion’s server code. v4 intends to replace the Python (and PyTorch) based server with a simple C++ version.

For Z-Image, the performance of the stock version of chromaForge is poorer than sd.cpp’s.

Worked on fixing Z-Image support in ED’s fork of chromaForge (a fork of Forge WebUI).

Collecting the worklog over the past few weeks.

The new engine that’ll power Easy Diffusion’s upcoming v4 release (i.e. sdkit3) has now been integrated into Easy Diffusion.

Managed to get stable-diffusion.cpp integrated into sdkit v3 and Easy Diffusion. sdkit v3 wraps stable-diffusion.cpp with an API server.

Following up to the previous post on sdkit v3’s design: The initial experiments with generating ggml from onnx models were promising, and it looks like a fairly solid path forward.

Successfully compiled the VAE of Stable Diffusion 1.5 using graph-compiler.

PolyBlocks is another interesting ML compiler, written using MLIR.

Wrote a simple script to convert ONNX to GGML.

Following up to the deep-dive on ML compilers: sdkit v3 won’t use general-purpose ML compilers.

This post concludes (for now) my ongoing deep-dive into ML compilers, while researching for sdkit v3.

A possible intuition for understanding GPU memory hierarchy (and the performance penalty for data transfer between various layers) is to think of it like a manufacturing logistics problem: CPU (host) to GPU (device) is like travelling overnight between two

Good post on using MLIR for compiling ML models to GPUs.

Wrote a fresh implementation of most of the popular samplers and schedulers used for image generation (Stable Diffusion and Flux) at https://github.com/cmdr2/samplers.cpp.

Some notes on machine-learning compilers, gathered while researching tech for Easy Diffusion’s next engine (i.e. sdkit v3).

The design constraints for Easy Diffusion’s next engine (i.e. sdkit v3) are: Lean: Install size of < 200 MB uncompressed (excluding models).

Cleared the backlog of stale issues on ED’s github repo.

Experimented with TensorRT-RTX (a new library offered by NVIDIA).

Development update for Easy Diffusion - It’s chugging along in starts and stops.

Upgraded the default version of Easy Diffusion to Python 3.9.

Easy Diffusion (and sdkit) now also support AMD on Windows automatically (using DirectML), thanks to integrating with torchruntime.

Spent the last week or two getting torchruntime fully integrated into Easy Diffusion, and making sure that it handles all the edge-cases.

Continued to test and fix issues in sdkit, after the change to support DirectML.

Worked on adding support for DirectML in sdkit.

Continued from Part 1.

Continued in Part 2, where I figured out how to include the required libraries in the wheel.

Spent the last few days writing torchruntime, which will automatically install the correct torch distribution based on the user’s OS and graphics card.

Spent most of the day doing some support work for Easy Diffusion, and experimenting with torch-directml for AMD support on Windows.

Spent a few days prototyping a UI for Easy Diffusion v4.

Notes on two directions for ED4’s UI that I’m unlikely to continue on.

Worked on a few UI design ideas for Easy Diffusion v4.

Spent some more time on the v4 experiments for Easy Diffusion (i.e.

Spent a few days getting a C++ based version of Easy Diffusion working, using stable-diffusion.cpp.

tl;dr - Today, I worked on using stable-diffusion.cpp in a simple C++ program.

tl;dr: Explored a possible optimization for Flux with diffusers when using enable_sequential_cpu_offload().