lua-openai
lua-openai copied to clipboard
Make _request response more useful when stream: true
(On top of #10)
This PR is created to discuss possible OpenAI\_request refactoring.
What's changed:
- Collect chunks during streaming, and instead of bare text,
return synthetic response object, similar to one returned when
stream: false - Simplify
ChatSession\generate_response, as there is no longer a need to re-parse the response strings
Reasoning:
- I argue that a simple plain-text return value proves to be quite ineffective when dealing with multiple SSE events. Consequently, it is more beneficial to allow the callback function to procure the entirety of the parsed data.
- Despite this, receiving integrated data as a return value might still be quite convenient. Use Case: The callback function primarily serves to display streaming output to the user. However, after this process, it becomes simpler to handle the integrated data.
The 1st one is most important, and I consider it as necessity. And I would like to know your opinion on the 2nd one.
To be done:
- [x] Collect other properties, such as
finish_reason,usage, etc.
Done in the last commit.
Cases where n>1 have also been tested.