llama-shepard

Results 6 comments of llama-shepard

Improve OpenAI API compatibility

@jeffreyftang Chat Templates (https://huggingface.co/docs/transformers/chat_templating) will handle this already, right? But some models do not support tools in their default Chat Templates (Like Llama 3)

Use special tokens specific to the fine-tuned adapter during decoding

Will this be completed? Planning to use adapters with special tokens like the below ones: https://huggingface.co/Dogge/llama-3-8B-instruct-Bluemoon-Freedom-lora/ https://huggingface.co/Dogge/llama-3-70B-instruct-uncensored-lora

feat: Function calling with output schema enforcement

@tgaddair @magdyksaleh Waiting for this merge to try out Function Calling in Lorax

feat: Function calling with output schema enforcement

@magdyksaleh Still waiting for this feature. I want to deploy Mistral Nemo and Llama 3.1 models with function calling

feat: Function calling with output schema enforcement

I have been waiting for function calling feature for long time and got exited for this PR. But this PR is still not merged. @tgaddair @jeffreyftang

[RFC]: Enhancing LoRA Management for Production Environments in vLLM

I would love to add the following feature in this RFC. ### LOAD ADAPTERS FROM S3 SUPPORTED STORAGES LoRAX has this feature [https://loraexchange.ai/models/adapters/#s3](url) This brings a new challenge in vLLM....