serving
serving copied to clipboard
[Feature Request] Support optional named input fields and null values for REST API
Feature Request
If this is a feature request, please fill out the following form in full:
Describe the problem the feature is intended to solve
Currently TF Serving does not provide full support for optional named input or null values for REST APIs:
- Optional named input: all named input exported by SavedModel serving signature must present in the REST request payload, otherwise it will trigger HTTP 400 error and the code path is here code
- REST null values: does not support by any tf.data types and casting will fail at runtime
Describe the solution
A clear and concise description of what you want to happen - in both cases (missing named input or passing in null value in the request body), TFS should be able to take a default value or leave the handling to serving functions.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered. N/A
Additional context
Add any other context or screenshots about the feature request here. Sample error log for missing named input request:
2022-01-07 21:24:06.650327: I tensorflow_serving/model_servers/http_server.cc:162] Processing HTTP request: POST /v1/models/model-name:predict body: 233767 bytes.
2022-01-07 21:24:06.651157: I tensorflow_serving/model_servers/http_server.cc:194] Error Processing HTTP/REST request: POST /v1/models/model-name:predict Error: Invalid argument: Missing named input: missing_feature in 'inputs' object.