Intel Neural Compute Stick 2 support

[TazDevl74]

I know it's old... but would it be possible to support this (on linux/debian)?

[AlexCheema]

Do you have one? What's your use case?

[TazDevl74]

Usecase so far unknown, I bought one due to a severe case of gadgettitus.

[Evanev7]

Would love to! No idea what support looks like though - I imagine Vulkan is the way to go but no research done yet.

[JakeHillion]

I'm not sure if I'm missing anything, so please correct me if I'm wrong. It looks like the Compute Stick 2 has 512MiB of RAM and really not very much compute, along with a slow interface to the host machine. I'm not sure how useful this will be for EXO (or LLMs in general) acceleration unfortunately, this puts it in much the same space as the Google Coral TPU where it's good at accelerating small kernels for images that fit on the device but bad for much larger concepts like LLMs.