deep-learning-containers icon indicating copy to clipboard operation
deep-learning-containers copied to clipboard
verified publisher icon aws

[feature-request] Support for PyTorch EI 1.7

Open luispsantos opened this issue 4 years ago • 1 comments

Checklist

  • [x] I've prepended issue tag with type of change: [feature]
  • [ ] (If applicable) I've documented below the DLC image/dockerfile this relates to
  • [ ] (If applicable) I've documented the tests I've run on the DLC image
  • [x] I'm using an existing DLC image listed here: https://docs.aws.amazon.com/deep-learning-containers/latest/devguide/deep-learning-containers-images.html
  • [ ] I've built my own container based off DLC (and I've attached the code used to build my own image)

Concise Description:

The latest supported Docker image for PyTorch with elastic inference is 1.5.1. This places us at a disadvantage when trying to use elastic inference containers to deploy recent PyTorch models which require version ≥ 1.7. Is there any ETA for adding 1.7 or 1.8 support for EI Docker images?

Describe the solution you'd like PyTorch docker images for EI using 1.7 or 1.8 version.

Describe alternatives you've considered I considered installing the latest EI PyTorch binaries in a custom Docker image, but it's not clear to me where those binaries live.

luispsantos avatar Oct 20 '21 07:10 luispsantos

I would love to upvote this feature request as currently I am struggling with getting EI to work on PyTorch 1.12.0

yamazakiji avatar Aug 25 '22 04:08 yamazakiji

We are going to deprecate Elastic Inference by end of April, 2024. If you are currently using Elastic Inference, you may get better performance at a much better price than Amazon EI with new hardware accelerator options such as AWS Inferentia or the GPU accelerated ml.g4dn.xlarge for your machine learning (ML) inference workloads. Please consider migrating your workloads to options that offer better price and performance.

Other benefits of using alternatives

  • Amazon Elastic Inference only supports Pytorch 1.5 and Tensorflow 2.3. If you migrate to ml.g4dn instances, you can use the latest versions of Pytorch 1.11 and Tensorflow 2.9
  • ml.g4dn and AWS Inferentia are available in all regions, where as Amazon Elastic Inference is only available in 5 regions.
  • Both AWS Inferentia and ml.g4dn offer better performance at lower price for most ML inference workloads.

See: https://docs.aws.amazon.com/sagemaker/latest/dg/ei.html

pravali96 avatar Feb 26 '24 21:02 pravali96