vkc1vk comments

Repositories
Issues
Comments

Results 4 comments of


                                            vkc1vk

[Feature] plan to support medusa?

Hi I was wondering Medusa will be supported with full tree attention or the Top-1 version currently available in vLLM? Thanks. cc: @zhyncs @merrymercy

Is ReDrafter supported by the TensorRT-LLM backend?

Referring to this -> https://github.com/NVIDIA/TensorRT-LLM/blob/v0.13.0/examples/redrafter/README.md

Is ReDrafter supported by the TensorRT-LLM backend?

@byshiue could you confirm if support will be added? Thanks!

Empty outputs with TRT engine built with W4A8, FP8 KV cache

I'm seeing this issue in v13, is there an ETA on the fix? Thanks @Barry-Delaney