Lan Gong

Trigger CI builds for pytorch/pytorch and domain libs in pytorch/builder via inter repository dispatch

3

This is the first step to synchronize and consolidate pytorch repository builds in the builder repo: - `pytorch/builder` will send a dispatch event to one of `pytorch/pytorch`, `pytorch/vision`, `pytorch/test`, `pytorch/audio`...

Multi Instance Execution Does Not Improve Throughput for YOLOv8 Model

1

**Description** I use Triton through KServe to deploy YOLOv8 model with dynamic batching and multi-instance execution enabled. I notice that using 2 model instances only marginally improves throughput. Since the...

CPU Throttling when Deploying Triton with ONNX Backend on Kubernetes

6

**Description** I am deploying a YOLOv8 model for object-detection using Triton with ONNX backend on Kubernetes. I have experienced significant CPU throttling in the sidecar container ("queue-proxy") which sits in...

Issue with FP16 Output for Half Precision Model

10

**Description** I am running a half-precision ONNX model inside Triton with FP16 data type for both the input and output. I have a downstream component (in a separate Kubernetes pod...