refinery icon indicating copy to clipboard operation
refinery copied to clipboard

Do not use pipelines sequentially on GPU for zero-shot calculation

Open FelixKirsch opened this issue 3 years ago • 0 comments

Is your feature request related to a problem? Please describe. When running a zero-shot model on GPU a warning is emitted. Screenshot 2022-11-15 at 18 01 53 The zero-shot services run one pipeline after another on batches of records. This is not optimal and decreases the performance of the calculation.

Describe the solution you'd like The huggingface documentation describes different ways to implement a solution so that the pipelines are run in parallel.

FelixKirsch avatar Nov 15 '22 17:11 FelixKirsch