server icon indicating copy to clipboard operation
server copied to clipboard

Expose IsLastResponse of InferResponse to the Python API

Open danilaeremin opened this issue 1 year ago • 0 comments

Is your feature request related to a problem? Please describe. It's not possible to understand in BLS models if current response of decoupled model is final. C++ API has IsLastResponse method, however it's not exposed to the Python (despite it's Python backend repository). Such interface requires to create not the best variant of code (creating code duplication or adding more complex conditions or pass some extra custom flags from the model). Mostly during processing sequence responses it's important to know if meaningful part of the result is ready or it's the end. Such method can solve that missing part of the condition.

Describe the solution you'd like Expose function IsLastResponse of InferResponse to the Python interface via .def("is_last_response", &InferResponse::IsLastResponse); in src/pb_stub.cc of PythonBackend repository.

Describe alternatives you've considered Currently it's possible to know, that the response was last only after loop, it sometimes requires some post-processing, which can duplicate code or create some extra complications. Second alternative is to pass flags via parameters or via extra tensor. That solutions are custom and is not the best way if standard FINAL flag exists.

danilaeremin avatar Apr 24 '25 20:04 danilaeremin