Options in server_core.c are not accesible neither from model_config_list or as a flag

[Sergiodiaz53-Dedalus]

Hello,

There seems to be a known behaviour of infinite looping of trying to load a model when a model_path is not correct.

To avoid that behaviour, I have found there are two options in server_core:

in 2.14.1 (AFAIK), we have servable_versions_always_present and in 2.18 (AFAIK) we have should_retry_model_load. The problem is that neither of them are accesible using a flag (you can check main.cc ) when launching the tensorflow_model_server. Meaning I can't run something like:

tensorflow_model_server --model_config_file=/tf/models/models.config --servable_versions_always_present=true

but there isn't neither a way of make use of them through the model_config_list. At least not one that is documented. I have tried things like:

model_config_list { config { name: "my_model" base_path: "/models/my_model/model" model_platform: "tensorflow" } servable_versions_always_present: true

or

model_config_list { config { name: "my_model" base_path: "/models/my_model/model" model_platform: "tensorflow" servable_versions_always_present: true }

For example, with any luck. In the documentation there is not mention neither to this.

I would really appreciate if someone can clarify whether this is a bug (options that can't be set up) or if it is a lack of documentation and in fact there is a way of setting this up through the model_config_list or any other way. In both cases I can try to contribute to fix that.

Thanks in advance.

[janasangeetha]

Hi @Sergiodiaz53-Dedalus Thank you for reporting. I'll investigate and provide an update here.

[janasangeetha]

Hi @Sergiodiaz53-Dedalus Yes, we are unable to use servable_versions_always_present and should_retry_model_load flags as they are unavailable in main.cc. I will check with the team internally if we are planning to pick this activity. Will provide an update here. Thank you!