⚡️🎉A PyTorch-native Inference Engine with Cache,
Parallelism, Quantization and CPU Offload for DiTs

Overviews

🤗Why Cache-DiT❓❓Cache-DiT is built on top of the 🤗Diffusers library and now supports nearly ALL DiTs from Diffusers. It provides hybrid cache acceleration (DBCache, TaylorSeer, SCM, etc.) and comprehensive parallelism optimizations, including Context Parallelism, Tensor Parallelism, hybrid 2D or 3D parallelism, and dedicated extra parallelism support for Text Encoder, VAE, and ControlNet.

Cache-DiT is compatible with compilation, CPU Offloading, and quantization, fully integrates with SGLang Diffusion, vLLM-Omni, TensorRT-LLM, ComfyUI, and runs natively on NVIDIA GPUs, Ascend NPUs and AMD GPUs. Cache-DiT is fast, easy to use, and flexible for various DiTs (online docs at 📘cache-dit.io).

📊Examples - The easiest way to enable hybrid cache acceleration and parallelism for DiTs with cache-dit is to start with our examples for popular models: FLUX, Z-Image, Qwen-Image, Wan, etc. ❓FAQ - Frequently asked questions including attention backend configuration, troubleshooting, and optimization tips

Table of contents

• 📘Overviews
• 📦Installation
• 🚀Quick Examples
• 🛠️Unified Cache APIs
• 🏗️DBCache Design
• 🔄Context Parallelism
• ⚡️Tensor Parallelism
• 🧩TE-P, VAE-P and CN-P
• 📐2D and 3D Parallelism
• 💾Low-Bits Quantization
• 🧠Attention Backends
• 🔥Use Torch Compile
• 🚀Use CUDA Graphs
• 🌐Use Ray Wrapper
• 🛠️Layerwise Offload
• 🛰️Ascend NPU Support
• 🖥️AMD GPU Support
• 📄Config with YAML
• 🌐Environment Variables
• 🚀Serving Deployment
• 📊Metrics Tools
• 📈Profiler Usage
• 📚API Documentation
• 📊Supported Matrix
• 📈Benchmark
• 👨‍💻Developer Guide
• 🌐Community Integration
• ❓FAQ