optimum-graphcore icon indicating copy to clipboard operation
optimum-graphcore copied to clipboard

Allow the encoder outputs to be computed on the CPU for generation

Open michaelbenayoun opened this issue 3 years ago • 0 comments

Currently, during generation, the encoder outputs can only be computed after having compiled the encoder separately. It would be nice to be able to compute the encoder outputs directly from the CPU, and to run the decoding loop on the IPUs. The IPUConfig attribute for that, and some other features have already been added, but the whole thing does not work for now.

michaelbenayoun avatar Apr 25 '22 08:04 michaelbenayoun