Cameron Quilici
Cameron Quilici
@jthomson04 > Our current sgl and trtllm (once that gets merged) disagg submissions use unique launch scripts, which will complicate the stated goal of upstreaming our scripts to the this...
@Oseltamivir Can you pls fix the merge conflicts so I can review more holistically ?
we're gonna hold off on this til #251 gets merged this week
@jgangani so sorry brother but can you please rebase with main following the convention set forth in https://github.com/InferenceMAX/InferenceMAX/pull/251 ?
@jgangani hi! where are we on this?
old perf Model | Hardware | Framework | Precision | ISL | OSL | TP | EP | DP Attention | Conc | TTFT (ms) | TPOT (ms) | Interactivity...
pls also correct PR description
Hi @jefverschuerend09 Absolutely, this is currently being developed.
> ``` > ImportError: Please install vllm[video] for video support > ``` > > Please read the error message. Hi. I am using the official vLLM image which states support...
@TheFloHub What is the final HTTP request body that you formulated that ended up working?