epfml
explore WebGPT for LLMs
explore the WebGPT library as a potential (seems full?) replacement for TFJS. try on small standard architecures like nanoGPT
we can explore this in parallel to our tfjs based prototype #641
who would be interested in giving this a shot? (can first be tried outside disco). maybe robin / albert potentially?
WebGPT is currently only compatible with Chrome Canary. It seems to be a big restriction for DISCO to have, what do you think @martinjaggi?
no issue at the moment (huge audience for LLMs, and huge audience of chrome, so sufficiently many people interested)
to add to @JulienVig's answer, it's an expertimental, still in draft, technology that's not supported around. there is currently only 25% of support for WebGPU, 0% on mobile. that effectively drops support for most of our targetted audience.
there might be way to optionally support it (isomorphic-webgpu that'll write ourself) but I don't know how well WebGPT supports it.
there are various backend of tfjs: WebGL (≠ WebGPU), wasm, cpu; the first one is the one running in the webapp (at least on my stable firefox). there is an ongoing project to add a WebGPU backend to tfjs which we can try out and would allow us to fallback on a slower backend if not supported.
all in all, I don't see the point of adding WebGPT for the only benefit of faster computation. are there other reasons maybe? (models based on it that's are not trivially converted to tfjs for eg?)
yea best would be to optionally support (only for LLM tasks, not affecting the rest of disco), and if not available just show an error message to the user