Matan Kleyman comments

Results 20 comments of


                                            Matan Kleyman

Per-class AP for specific classes and new datasets

@JaydenKing32 Can you please describe the method that worked for you ? Keeping h.num_classes as coco default and only change h.label_map to your custom dataset ? Thanks

Would it be possible to support LoRA fine-tuned models?

As far as the hotswap is still not implemented.. What is the current best way to run lora weights using vllm ? Should I merge the lora weights and the...

feat: Pandas v2 compatibility

Hi guys, thanks for the great work. Do you know when is this planned to be released ? I see it was merged but no release was published yet.

GPU's Strategy Slow Model Intialization

This is the paramters config file: ` num_classes: 99 anchor_scale: 1.0 label_map: {*****, dict of 98 classes'}`

Support for Pydantic v2

Allow deletion of models, bentos, and deployments via yatai UI

I'm interested in the ability to delete models and bentos as well. @eledhwen Did you find any workaround ? @parano Is it still on the roadmap ?

Support generation from input embedding

This PR would be super valuable for us. @pfldy2850 Do you plan to adjust it to the current master branch ? Because I see it is a bit outdated.

bug: 40GB is not enough for llama-2-7b

Any updates on how to fix that? facing the same issue when running mistral-7b quantized 4 bit with --backend vllm. @pavel1860 @aarnphm @1E04 @marijnbent @Bernsai @dudeperf3ct

[Usage] Unable to load LLaVA v1.6 models

@haotian-liu I try to run the 4-bit 34B on 24GB Ram but I'm pretty sure it offloads some of the weights to cpu, because of `low_cpu_mem_usage=True` which results in the...

[Usage] llava-v1.6-mistral-7b will load in the demo, but llava-v1.6-34b will not.

Same issue here. Were you able to fix that ? @levi @iceman-p I suspect this is related to `device="auto"` and `low_cpu_mem_usage=True`