rohan902

How to use endpoint for model published on EC-2 server

2

Hi, I was wondering if we use custom endpoint for LM. I have published my model on an ec2-machine and calling my model vi API through endpoint to do inference...

"error":{"code":500,"message":"rpc error: code = Unknown desc = unimplemented","type":""}}

15

**LocalAI version:** Latest **Environment, CPU architecture, OS, and Version:** EC-2 **Describe the bug** Getting the grpc connection error when running using cuda12 image. But when running through Vanilla/cpu image, its...