Vipul gote

icon indicating a website link https://sites.google.com/view/vipul-ai/home?read_current=1

icon company Tech Mahindra icon location pune icon location Software Engineer | Currently working on LLM's, Machine Learning and BigData analytics

Results 2 comments of Vipul gote

[Feature]: Consider parallel_tool_calls parameter at the API level

+6

[Bug]: OOM when running llama3.1-8B-Instruct

Hi , Few suggestion you should use quantize version...llama3.1 8B itself has size of over 16GB....so it is not going to fit in your gpu Still u can use it...