Prajwal A
Prajwal A
For https://github.com/vllm-project/vllm-omni/issues/175 ## Purpose Integrate TeaCache (Timestep Embedding Aware Cache) into `vllm-omni` to speed up diffusion inference (~1.5–2x) with minimal quality loss by reusing transformer block computations when consecutive timestep...
# Add Flux 2.0 diffusion model support Refer issue: https://github.com/vllm-project/vllm-omni/issues/153 ## Purpose Add support for Flux 2.0 text-to-image diffusion model with dual-stream (8 blocks) + single-stream (48 blocks) transformer architecture....
### Motivation. Add TeaCache (Timestep Embedding Aware Cache) to vllm-omni to accelerate diffusion model inference by 1.5x-2x with minimal quality degradation. TeaCache reuses transformer block computations when consecutive timestep embeddings...
# What does this PR do? ### What is TeaCache? [TeaCache](https://github.com/LiewFeng/TeaCache) (Timestep Embedding Aware Cache) is a training-free caching technique that speeds up diffusion model inference by **1.5x-2.6x** by reusing...
### Motivation While validating large diffusion models in vLLM-Omni (e.g., during [https://github.com/vllm-project/vllm-omni/pull/302](https://github.com/vllm-project/vllm-omni/pull/302)), model initialization and execution required CPU offloading even on high-memory GPUs (e.g., H100). More generally, CPU offloading also...
## Purpose Add CPU offloading support for all diffusion models in vllm-omni. This enables memory-efficient inference by automatically moving model components (text encoder, DIT transformer, VAE) between CPU and GPU...
# Add FLUX 2 diffusion model support Ref: [`vllm-project/vllm-omni#153`](https://github.com/vllm-project/vllm-omni/issues/153) ## Summary Adds support for **FLUX 2** text-to-image diffusion with a dual-stream + single-stream transformer architecture. ## What’s included - **`Flux2Transformer2DModel`**:...
PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED. ## Purpose Refer https://github.com/vllm-project/vllm-omni/issues/412 ## Test Plan ## Test Result --- Essential Elements...