sagemaker-distribution
sagemaker-distribution copied to clipboard
Python 3.11 Support to sync with AL2023
🚀 The feature, motivation and pitch
I connect to EMR 7.1 from sagemaker using spark connect, this would be more straight-forward if both were using the same version of python AL2023 has default support for 3.9 and 3.11 but not 3.10 which is what sagemaker distribution is on, could we update the image to 3.11
Use case description
Spark Connect UDFs are not possible
Alternatives
I could install 3.10 manually using bootstraps, but this is slows down node start up time
Additional context
No response
We are upgrading to Python 3.11 in our 2.0 release
https://github.com/aws/sagemaker-distribution/pull/457
v2.0 image has been released with Python 3.11 support: https://gallery.ecr.aws/sagemaker/sagemaker-distribution