Expose streaming API

Open lpil opened this issue 1 year ago • 5 comments

For example, for @hayleigh-dot-dev's chat-bot project

Jul 14 '24 08:07 lpil

hey I'm also interested in a response body streaming API, did you have a particular design in mind?

Maybe something like this?

pub fn send_stream(req: Request(String)) -> Result(Response(Iterator(BitArray)), Dynamic)

Oct 19 '24 07:10 RobertCraigie

I think it would be message based. Iterators are not suitable for streaming like this.

Oct 20 '24 10:10 lpil

ah I see thanks, would you be able to share an example snippet? I'm new to Gleam so I'm not sure what that would look like exactly

Oct 25 '24 17:10 RobertCraigie

I don't believe anyone has done any research and design on this yet.

Oct 28 '24 14:10 lpil

+1 for this, was playing around with Ollama and would like to stream the response

Jun 18 '25 09:06 nathanjcollins