Vaibhav Hiwase issues

Repositories
Issues
Comments

Results 3 issues of


                                            Vaibhav Hiwase

Redis callbacks are added and demonstrated using webhook container

Please find the postman collection for understanding features. Try running postman APIs (named POST async) without "webhook_endpoint" payload key and take a look at the logs for understanding how Redis...

Update in inference.py -- omitting the optimizer's state

During the inference phase when loading a trained model, there's a focus on exclusively loading the model's weights while disregarding or not loading the optimizer's state. This adjustment can be...

Exception: CUDA error: an illegal memory access was encountered. CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.

I attempted to serve the original base model of **Llama 3.1** in 4-bit, both with and without setting `load_in_4bit`. Below are my observations. When `load_in_4bit = True`: The model throws...

currently fixing