Handling text generation edge cases

[cyyeh]

• [ ] generated tokens reach token window limitation according to respective llm model specs.
• [ ] (nice to have) add moderation layer to make our app more safe. reference: https://platform.openai.com/docs/guides/text-generation/how-can-i-make-my-application-more-safe

[cyyeh]

reference:

https://spacy.io/api/large-language-models#task-sharding