InferenceClient: provide generator-based bulk inference

Open mhaas opened this issue 5 years ago • 0 comments

The current implementation will try to perform inference for all chunks. If an exception occurs, the progress is lost.

By yielding the individual chunks, the caller can be in charge of error handling.

We can either change the existing API or introduce a separate method (and implement do_bulk_inference using the new method).