omer-dayan

Results 4 comments of omer-dayan

[feature] Dynamic MIG partitioner

I agree with @klueska about how DRA is the right way. However, @elgalu, I do not agree its not feasible under the current circumstances. You welcome to watch the following...

Scaling any scalable RD

My use case is KPA. I want to be able to scale a statefulset, which is not kNativeService, using KPA

[Bug]: Audio transcription cannot load `preprocessor_config.json` when using runai streamer

Hey @gilljon ! From the temporary directory I can understand you store the model files in S3, right? We pull all the metadata files from the S3 to a tmp...

Loading models from an S3 location instead of local path

Hey, At RunAI we had published an open source tool to stream model weights from an object store like S3 to GPU memory - called RunAI Model Streamer (https://github.com/run-ai/runai-model-streamer) The...