marcoripa96

Stream tokens output

15

Is it possible to stream each token of the output as soon as it is generated by the model? I guess it depends on the hugging face transformers classes and...

Is there any way to support multiple parallel generation request to the same model?

11

I'm kind of a newbie and probably it's not the right thing to ask, but maybe I can get pointed to the right direction. I have a FastAPI server and...

### Description When reading the the documentation of createStreamableUI I see the following: - The `append `method is used to append a new UI node to the end of the...

ai/rsc components streamed through createStreamableUI are mounted multiple times when updating the ui node (update/done)

5

### Description I was reading through the https://chat.vercel.ai/ github example and in particular I found this [useStreamableText](https://github.com/vercel/ai-chatbot/blob/fa9f0947f0a7983cf5022cbbc1416910349dd5e4/lib/hooks/use-streamable-text.ts) hook. I report the implementation here for clarity: ```tsx import { StreamableValue, readStreamableValue...