lightvector
[feature request] supporting multiple backends simultaneously
Hi.
Currently, KataGo has common interface for backends, and the cache variable USE_BACKEND determines which implementation is used.
On Apple Silicon, two (or three) backends, Neural Engine, GPU, (and CPU), are available simultaneously. Android, and now Copilot+ PC are in the same situation, they have GPU, and NPU.
Superclass-subclass implementation will be preferable for such devices.
In fact, I had tried it and worked. But, I had given up to modify batch size related code. Each backend should have independent batch size, and it is needed to modify several parts. Using suitable batch-size on each backend is essential to get good performance.
I may try again. I will be happy if someone will be interested in it. Thanks.
I'm very interested in NPU backend. My CPU is Intel Core Ultra which have a NPU, but it have never been able to work. In your statement, you had successfully developed those backends? If so, would you like to release it?
