sagemaker-inference-toolkit icon indicating copy to clipboard operation
sagemaker-inference-toolkit copied to clipboard

Launch MMS without repackaging model contents

Open fm1ch4 opened this issue 3 years ago • 0 comments

Describe the feature you'd like Inference toolkit, when starting up MMS, will repackage the model contents by copying the contents from /opt/ml/model to /.sagemaker/mms/models: https://github.com/aws/sagemaker-inference-toolkit/blob/master/src/sagemaker_inference/model_server.py#L76.

This is unnecessary and MMS can simply read the model contents from /opt/ml/model. This will save some startup time due to removing the need to copy files from one location to another, and will also help the container restart successfully as it will not run into the issue where /.sagemaker/mms/models/model is already present (say if the container crashed and is restarted on the same host).

How would this feature be used? Please describe. See above

Describe alternatives you've considered N/A

Additional context N/A

fm1ch4 avatar Mar 28 '22 21:03 fm1ch4