Input preprocessing for inference call inside Sagemaker

[nahidalam]

I have a pretrained Keras model (tensorflow backend) - model.json and model-weight.hdf5 which I was able to deploy using this tutorial. The tutorial converts those files to protobuf format and creates model.tar.gz and puts it in S3

I can call the deployed endpoint from a boto client code in my local machine. The client code does some preprocessing before calling the endpoint. I need to put this preprocessing portion of the code inside Sagemaker. I am assuming I need to update the entry point file train.py to have the preprocessing code. But I am not sure how. Should I override input_fn? If so how?

Below is the client side preprocessing code that I need to put inside sagemaker instead of a separate client

img_path = 'shoe.jpg' # path to a local image downloaded from S3
img = image.load_img(img_path, target_size=(256, 256))
img_data = image.img_to_array(img)
img_data = np.expand_dims(img_data, axis=0)
img_data = preprocess_input(img_data)
data = img_data.tolist()

Right now, I invoke the endpoint like below which I understand that the Body portion will change to a JSON payload of S3 data download link.

response = client.invoke_endpoint(EndpointName=endpoint_name, Body=json.dumps(data))

[nadiaya]

Here is documentation on how to do this using sagemaker-python-sdk: https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/tensorflow/deploying_python.rst#overriding-input-preprocessing-with-an-input_fn

[gerdm]

Hi @nadiaya,

I am stuck with the same problem. I went into the given url, but there does not seem to be a specific way on how to override input_fn. What does it mean to add input_fn to my training script? I've trained my model outside sagemaker.

thanks in advance.

[jmsanchezo]

Hi, We are in the same situation. We have created the model without problems but we can´t override the input_fn or predict_fn. AWS is executing the default function. Someone can help us @nadiaya, @gerdm Thanks in advance.

This is the error: Exception on /invocations [POST] Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/sagemaker_containers/_functions.py", line 85, in wrapper return fn(*args, **kwargs) File "/usr/local/lib/python3.5/dist-packages/sagemaker_sklearn_container/serving.py", line 54, in default_input_fn return np_array.astype(np.float32) if content_type in content_types.UTF8_TYPES else np_array TypeError: float() argument must be a string or a number, not 'dict' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 2311, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1834, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1737, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 36, in reraise raise value File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1832, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1818, in dispatch_request return self.view_functions[rule.endpoint](**req.view_args) File "/usr/local/lib/python3.5/dist-packages/sagemaker_containers/_transformer.py", line 171, in transform result = self._transform_fn(self._model, request.content, request.content_type, request.accept) File "/usr/local/lib/python3.5/dist-packages/sagemaker_containers/_transformer.py", line 197, in _default_transform_fn data = self._input_fn(content, content_type) File "/usr/local/lib/python3.5/dist-packages/sagemaker_containers/_functions.py", line 87, in wrapper six.reraise(error_class, error_class(e), sys.exc_info()[2]) File "/usr/local/lib/python3.5/dist-packages/six.py", line 692, in reraise raise value.with_traceback(tb) File "/usr/local/lib/python3.5/dist-packages/sagemaker_containers/_functions.py", line 85, in wrapper return fn(*args, **kwargs) File "/usr/local/lib/python3.5/dist-packages/sagemaker_sklearn_container/serving.py", line 54, in default_input_fn return np_array.astype(np.float32) if content_type in content_types.UTF8_TYPES else np_array

[jmsanchezo]

I answer myself. In our case we were deploying from boto3 the endpoint without previously executing the deletion of the previous endpoint configuration. Each time we deployed a new endpoint we did so by pointing to old code in which the functions input_fn and predict_fn did not yet exist. Thanks, I hope helps someone. (Python 3.5, Scikit, boto3)

[rgoomer]

Initialise your model like this - sagemaker_model = TensorFlowModel(model_data = s3Url, role = role, framework_version = '1.12', source_dir ='my_src', entry_point = 'preprocessing.py', env = {'SAGEMAKER_REQUIREMENTS': 'requirements.txt'})

Create folder 'my_src' and implement the preprocessing.py file with your code in the function - def input_fn(request_body, content_type). Also create a requirements.txt in 'my_src' which has your libraries like tf,keras,numpy etc.

[zuldeveloper2023]

Hi

Can someone help me with this? I created same function input_fn(req_body, content_type) in my entry .py file. But, when i invoke my endpoint, it fails. how do i override input_fn?

[zuldeveloper2023]

Hi @gerdm did you manage to solve it?