Enze Xie | Efficient AI

LongLive: Real-time Interactive Long Video Generation

Mon, 13 Oct 2025 00:00:00 +0000

Fast-dLLM v2: Efficient Block-Diffusion LLM

Tue, 30 Sep 2025 00:00:00 +0000

DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space

Mon, 29 Sep 2025 00:00:00 +0000

DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder

Mon, 29 Sep 2025 00:00:00 +0000

SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer

Mon, 29 Sep 2025 00:00:00 +0000

DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Space

Fri, 01 Aug 2025 00:00:00 +0000

DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer

Mon, 07 Jul 2025 00:00:00 +0000

Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding

Wed, 28 May 2025 00:00:00 +0000

SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation

Wed, 12 Mar 2025 00:00:00 +0000

VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation

Tue, 04 Mar 2025 00:00:00 +0000

SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer

Mon, 20 Jan 2025 00:00:00 +0000

SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

Thu, 07 Nov 2024 00:00:00 +0000

Overview

SVDQuant is a post-training quantization technique for 4-bit weights and activations that well maintains visual fidelity. On 12B FLUX.1-dev, it achieves 3.6× memory reduction compared to the BF16 model. By eliminating CPU offloading, it offers 8.7× speedup over the 16-bit model when on a 16GB laptop 4090 GPU, 3× faster than the NF4 W4A16 baseline. On PixArt-∑, it demonstrates significantly superior visual quality over other W4A4 or even W4A8 baselines. “E2E” means the end-to-end latency including the text encoder and VAE decoder.

Method

SVDQuant: Absorbing Outliers via Low-Rank Branch

Stage 1: Originally, both the activation $\boldsymbol{X}$ and weights $\boldsymbol{W}$ contain outliers, making 4-bit quantization challenging. Stage 2: We migrate the outliers from activations to weights, resulting in the updated activation $\hat{\boldsymbol{X}}$ and weights $\hat{\boldsymbol{W}}$ . While $\hat{\boldsymbol{X}}$ becomes easier to quantize, $\hat{\boldsymbol{W}}$ now becomes more difficult. Stage 3: SVDQuant further decomposes $\hat{\boldsymbol{W}}$ into a low-rank component $\boldsymbol{L}_1 \boldsymbol{L}_2$ and a residual $\hat{\boldsymbol{W}} - \boldsymbol{L}_1 \boldsymbol{L}_2$ with SVD. Thus, the quantization difficulty is alleviated by the low-rank branch, which runs at 16-bit precision.

Nunchaku: Fusing Low-Rank and Low-Bit Branch Kernels

(a) Naïvely running low-rank branch with rank 32 will introduce 57% latency overhead due to extra read of 16-bit inputs in Down Projection and extra write of 16-bit outputs in Up Projection. Nunchaku optimizes this overhead with kernel fusion. (b) Down Projection and Quantize kernels use the same input, while Up Projection and 4-Bit Compute kernels share the same output. To reduce data movement overhead, we fuse the first two and the latter two kernels together.

Performance

SVDQuant reduces the model size of the 12B FLUX.1 by 3.6×. Additionally, Nunchaku further cuts memory usage of the 16-bit model by 3.5× and delivers 3.0× speedups over the NF4 W4A16 baseline on both the desktop and laptop NVIDIA RTX 4090 GPUs. Remarkably, on laptop 4090, it achieves in total 10.1× speedup by eliminating CPU offloading.

Integrate with LoRA

SVDQuant seamlessly integrates with off-the-shelf LoRAs without requiring re-quantization. When applying LoRAs, it matches the image quality of the original 16-bit FLUX.1-dev.

Video

Acknowledgment

We thank MIT-IBM Watson AI Lab, MIT and Amazon Science Hub, MIT AI Hardware Program, National Science Foundation, Packard Foundation, Dell, LG, Hyundai, and Samsung for supporting this research. We thank NVIDIA for donating the DGX server.

Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models

Mon, 14 Oct 2024 00:00:00 +0000

HART: Efficient Visual Generation with Hybrid Autoregressive Transformer

Mon, 14 Oct 2024 00:00:00 +0000

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

Mon, 14 Oct 2024 00:00:00 +0000

Enze Xie | Efficient AI

LongLive: Real-time Interactive Long Video Generation

Fast-dLLM v2: Efficient Block-Diffusion LLM

DC-Gen: Post-Training Diffusion Acceleration with Deeply Compressed Latent Space

DC-VideoGen: Efficient Video Generation with Deep Compression Video Autoencoder

SANA-Video: Efficient Video Generation with Block Linear Diffusion Transformer

DC-AE 1.5: Accelerating Diffusion Model Convergence with Structured Latent Space

DC-AR: Efficient Masked Autoregressive Image Generation with Deep Compression Hybrid Tokenizer

Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding

SANA-Sprint: One-Step Diffusion with Continuous-Time Consistency Distillation

VILA-U: a Unified Foundation Model Integrating Visual Understanding and Generation

SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer

SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models

Overview

Method

SVDQuant: Absorbing Outliers via Low-Rank Branch

Nunchaku: Fusing Low-Rank and Low-Bit Branch Kernels

Performance

Integrate with LoRA

Video

Acknowledgment

Deep Compression Autoencoder for Efficient High-Resolution Diffusion Models

HART: Efficient Visual Generation with Hybrid Autoregressive Transformer

SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

Oral Presentation Video