Alex Cheema
Alex Cheema
Any updates here? @gauravsaini
Closing due to inactivity.
Hey @bayedieng just checking in. Anything I can help with to move this along?
How is this going @bayedieng? Anything I can help with?
> I started a branch using ggml given that the llama cpp api doesn't expose the weights and thus wouldn't be able to be sharded. What would then be the...
How's this going @bayedieng? Anything I can help with?
> Was waiting on confirmation for the bounty requirements. I should have a working llama implementation within the working week. Checking in again. You can use the PR for `TorchInferenceEngine`...
Hey - we did some work on homomorphic encryption for private search here: https://blog.exolabs.net/day-8 The main problem with doing it for model inference as you describe is the massive overhead....
> Got it. Thanks for the reply! > > Just to confirm, does each machine in the cluster know what prompt it’s processing and what the final output will be?...
Assigned $300 bounty to this