sagemaker-inference-toolkit icon indicating copy to clipboard operation
sagemaker-inference-toolkit copied to clipboard

Local development WorkerLifeCycle skip

Open danielmapar opened this issue 6 years ago • 1 comments

Is your feature request related to a problem? Please describe. Whenever I test my API locally I have to wait for about 5 min for the WorkerLifeCycle to finish. That slowdowns development immensely.

Describe the solution you'd like I would like a config of sorts to initialize the server faster (maybe a dummy worker) so I can test my inference function faster without waiting for 5 min

Describe alternatives you've considered Right now I made a separate Python script to simulate the call to my model (with the payload), but it is far from ideal

danielmapar avatar Feb 21 '20 16:02 danielmapar

You can overwrite MMS configuration through environment variables: https://github.com/awslabs/multi-model-server/blob/master/docs/configuration.md#environment-variables

Though I am not sure whether it's possible for WorkerLifeCycle and going to delegate this question to MMS team.

nadiaya avatar Feb 25 '20 22:02 nadiaya