Shawn Zhang

Results 7 comments of Shawn Zhang

@wenxzhen Could you please share some core code with ur or sent a PR to this repo?

Container SSA check-in. IHAC is running ML workloads with Inferentia on EKS. They are quite interested in Bottlerocket in terms of awesome security benefits they get with less overhead. They...

Thanks @kevin85421 . Steps include, 1. Setup the Prometheus and Grafana. follow this [doc](https://docs.ray.io/en/latest/cluster/kubernetes/k8s-ecosystem/prometheus-grafana.html) 2. In Grafana, import the dashboard as the snapshot. Copy and past the Json mentioned and...

Hi @vara-bonthu Darren works with me in a same team. One thing we have discussed, do you want to make this solution as a parallel pattern as others? or make...

propose to replace the Triton pattern, as NVIDIA side already redirect Triton to Dynamo now https://developer.nvidia.com/triton-inference-server

SGLang performs very well in deployment DeepSeek series models. Looking forward to see the SGLang engine support!