bottlerocket [EKS] Support Inferentia/Neuron Runtime

What I'd like: I think it requires the neuron driver on https://github.com/aws/aws-neuron-sdk

Any alternatives you've considered: Nothing

Mar 14 '22 16:03 samjo-nyang

Thanks for raising this. We're interested in integrating with Neuron, and it's something we're planning to look into down the road!

Mar 14 '22 17:03 cbgbt

Re-titled this to be consistent with #1075, which is similar but for an ECS Inferentia variant.

Mar 15 '22 22:03 cbgbt

Is this still needed?

Dec 19 '22 17:12 stmcginnis

Yes, we are using more neuron instances than I created the ticket. (actively migrating workloads from gpu to neuron)

Dec 20 '22 14:12 samjo-nyang

Container SSA check-in. IHAC is running ML workloads with Inferentia on EKS. They are quite interested in Bottlerocket in terms of awesome security benefits they get with less overhead. They really want to align the company standards to use Bottlerocket for general business application as well as ML workloads. But the lack of support for Inferentia would affect their adoption.

Sep 28 '23 13:09 hustshawn

IHAC who is running Stable Diffusion on EKS Inf2, and they wish to adopt Bottlerocket image cache solution to reduce the large image (10+GB) pulling time from ECR around 3-4 minutes. Foreseeing the increasing GenAI model hosting with Inferentia, supporting Inferentia/Neuron runtime will have a big impact.

Sep 28 '23 14:09 heichow