Why does model_management_pb2 imports tensorflow

[chuyang-deng]

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): ubuntu 18.04
• TensorFlow Serving installed from (source or binary): binary
• TensorFlow Serving version: 1.14

Describe the problem

I have a TensorFlow Serving container that is designed for serving purpose only. I want to load models at runtime without any delay so we implemented a gPRC client to send tensorflow_serving.apis.model_management_pb2.ReloadConfigRequest to our exposed gRPC port.

However, it looks like the model_management.proto file imports tensorflow/core/lib, which we do not want to have such dependency and such dependency should not exist here. Here's what I found from your code base:

• The comment above StatusProto linked to a thirdparty status.h which does not exist (https://github.com/tensorflow/serving/tree/master/third_party)
• The tensorflow_serving.apis BUILD doesn't build model_management.proto (https://github.com/tensorflow/serving/blob/master/tensorflow_serving/apis/BUILD)

Looks like if anyone wants to use this ReloadConfigRequest (or even anything that uses status.proto) I wonder if there's a workaround that I could do to avoid installing tensorflow. And it would be even better if you could decouple the dependency of tensorflow from tensorflow/serving.

Exact Steps to Reproduce

• Build a tensorflow serving only container without installing tensorflow, expose a gRPC port, run the server.
• Implement a gRPC client and send a tensorflow_serving.apis.model_management_pb2.ReloadConfigRequest to the server
• Error will occur (pasted in the next section)

Source code / logs

Include any logs or source code that would be helpful to diagnose the problem. If including tracebacks, please include the full traceback. Large logs and files should be attached. Try to provide a reproducible test case that is the bare minimum necessary to generate the problem.

from tensorflow_serving.apis import model_management_pb2 File "/usr/local/lib/python3.5/dist-packages/tensorflow_serving/apis/model_management_pb2.py", line 17, in <module> from tensorflow_serving.util import status_pb2 as tensorflow__serving_dot_util_dot_status__pb2 File "/usr/local/lib/python3.5/dist-packages/tensorflow_serving/util/status_pb2.py", line 16, in <module> from tensorflow.core.lib.core import error_codes_pb2 as tensorflow_dot_core_dot_lib_dot_core_dot_error__codes__pb2 ImportError: No module named 'tensorflow'

[chuyang-deng]

Hi @gowthamkpr,

Any updates？

Thanks, Chuyang

[christisg]

Hi @ChuyangDeng,

Did you install TF serving API? (pip install tensorflow-serving-api)

[chuyang-deng]

Hi @christisg,

Yes. tensorflow-serving-api, grpcio and protobuf are all installed.

[christisg]

If "pip install tensorflow-serving-api" completed successfully, it should have installed all the libraries you need (including the associated tensorflow libraries), and you should be able to find them under the local installation path. I'd recommend checking and confirming that your environment is set-up properly, specifically if Python is properly installed. If you're still unsure how to resolve the issue, please provide very concrete instructions on how to reproduce.

[chuyang-deng]

hi @christisg, thanks for the information. With successful pip install tensorflow-serving-api, I am able to find pb2 files such as classification_pb2.py, regression_pb2.py that are built from tensorflow/core library. However what is missing (and causing the error) is the error_codes_py2 that is used to built status_pb2.py (https://github.com/tensorflow/serving/blob/master/tensorflow_serving/util/status.proto#L5).

Are you suggesting that tensorflow binaries is expected to be installed?

[christisg]

You can find error_codes_pb2.py under tensorflow/core/lib/core (installed).

[chuyang-deng]

Hi @christisg, I understand that if we do pip install tensorflow-serving-api, tensorflow binaries will be installed as well. And then error_codes_pb2 can be imported successfully.

However, what I'm trying to do is pip install --no-dependencies --no-cache-dir tensorflow-serving-api, because I do not want to install tensorflow in a container that is for serving purpose only. (tensorflow binary is big, 109.2MB and I already had a container that's doing tensorflow training)

My question is why error_codes_pb2 is not built into tensorflow_serving/api as other pb2 files are.

[figroc]

@ChuyangDeng I have exactly the same issue as yours. In the end I set up a library to achieve that. May this helpful for you. https://github.com/figroc/tensorflow-serving-client

[chuyang-deng]

@ChuyangDeng I have exactly the same issue as yours. In the end I set up a library to achieve that. May this helpful for you. https://github.com/figroc/tensorflow-serving-client

Hi @figroc, thanks for responding. Yeah, I ended up doing the same thing. But this is definitely not a long-term solution. I'm still hoping the tensorflow team can come up with a solution or at least provide with an "official workaround".

[christisg]

Apologies for the delayed response. You're correct that we could completely decouple TF serving api from Tensorflow. To do this we'll need to basically duplicate the relevant dependencies in TF serving api, which might create conflicts for cases when Tensorflow is already installed on the same machine (for training for example). All of this is solvable, but will require some work, which is currently not prioritized. Let us know if you're interested in contributing.

[charlesoconor]

Couldn't you pull the proto buff definitions out of TensorFlow to a point where they could both be imported?

[singhniraj08]

@chuyang-deng,

Tensorflow Serving prediction APIs are defined as protobufs. Instead of loading Tensorflow and TF Serving depnedency, you can replace them by generating the necessary tensorflow and tensorflow_serving protobuf python stubs. This avoids the need the pull in the entire (heavy) Tensorflow library on the client itself. You can refer this article to implement this.

Please let us know if this helps you in using Tensorflow serving api without Tensorflow. Thanks.

[github-actions[bot]]

This issue has been marked stale because it has no recent activity since 7 days. It will be closed if no further activity occurs. Thank you.

[github-actions[bot]]

This issue was closed due to lack of activity after being marked stale for past 7 days.