llmaz
llmaz copied to clipboard
InftyAI
☸️ Easy, advanced inference platform for large language models on Kubernetes. 🌟 Star to support our work!
test CI
#### What this PR does / why we need it #### Which issue(s) this PR fixes Fixes # #### Special notes for your reviewer #### Does this PR introduce a...
#### What this PR does / why we need it ### Detailed Explanation of Commit This commit introduces a guide for configuring serverless environments on Kubernetes, focusing on integrating Prometheus...
https://github.com/InftyAI/llmaz/blob/3c13ff343e28cdbe842fd3c659e3daf0d1e27e91/.github/workflows/kube-workflow-init.yaml#L8 I want to move the workflow details to this repo.(currently in uses: kerthcet/github-workflow-as-kube/.github/workflows/[email protected]) And use https://github.com/jlumbroso/free-disk-space to do a clean up.
**What happened**: The pipeline has started failing unexpectedly in recent weeks. Previously stable workflows now encounter errors during execution, with no intentional changes made to the related code or configuration...
**What would you like to be added**: See discussion here: https://github.com/InftyAI/llmaz/pull/498#discussion_r2479021076 **Why is this needed**: **Completion requirements**: This enhancement requires the following artifacts: - [ ] Design doc - [...
#### What this PR does / why we need it Integrating Dragonfly for efficient model and Image distribution #### Which issue(s) this PR fixes Fixes #361 #### Special notes for...
#### What this PR does / why we need it #### Which issue(s) this PR fixes Fixes https://github.com/InftyAI/llmaz/issues/463#issuecomment-3036727238 #### Special notes for your reviewer #### Does this PR introduce a...
**What would you like to be added**: Right now, we need to apply the envoy ai gateway manually following the https://github.com/InftyAI/llmaz/tree/main/docs/examples/envoy-ai-gateway, however, we need to create all these stuff automatically...
**What would you like to be cleaned**: Envoy AI Gateway just released v0.2.0 days ago, which supports some features like customized ownedBy, created which is helpful to our project. Also,...
**What would you like to be added**: ```bash apiVersion: llmaz.io/v1alpha1 kind: OpenModel metadata: name: opt-125m spec: familyName: opt source: modelHub: modelID: facebook/opt-125m inferenceConfig: flavors: - name: h800 priority: 5 #...