Simon Mo

Yes. If multiple clients send request to the same model simultaneously (or in a high throughout situation), Clipper will send the largest batch possible to guarantee the latency slo. We...

Batch size is determined at https://github.com/ucbrise/clipper/blob/3c5a1cc6ce59e0ccd778f526a50808d0e7b2576f/src/libclipper/src/containers.cpp#L128 We use an estimation method to adaptively figure out the optimal batch size per model. The batching happen inside query frontend, which are the...

Can you clarify? Which part of Clipper do you want to debug? Your model log can be checked via docket log or kubectl log. I’m asking because Clipper has a...

One way to do it is to modify the dockerfile for QueryFrontend, instead of https://github.com/ucbrise/clipper/blob/3c5a1cc6ce59e0ccd778f526a50808d0e7b2576f/dockerfiles/QueryFrontendDockerfile#L17 You can write: ```dockerfile RUN apt-get install -y gdb ENTRYPOINT ["gdb", "/clipper/release/src/frontends/query_frontend"] ``` And build...

I'll leave the first question to @Corey-Zumar. key-value pairs are potential use case such that Clipper will directly be exposed as part of the micro-service. To answer the second question,...

Are you using kubernetes proxy? If so, you can pass in `kubernetes_proxy_addr` as an argument in constructing `KubernetesContainerManager` as described here: http://docs.clipper.ai/en/v0.3.0/container_managers.html#clipper_admin.KubernetesContainerManager.__init__

Thanks for the suggestion @csmithnm! We are slightly behind on R support and these feedback are great! I will work on fixing these soon.

Yes. Please send a PR if you are interested! Thanks a lot!

Prometheus can track these with node exporter https://github.com/prometheus/node_exporter/blob/master/README.md For Kubernetes we can just scrape kube-api-server/metrics. Kubernetes expose Prometheus metrics there

Update: - For Docker, we can safely assume user only has on node so we can just run a node exporter at startup. - Kubernetes's API service does expose metrics...