Models for text classification & summarisation, and running on CPU

[Dee-Ma]

Hi,

Many thanks for releasing this library for using LLMs! We just have two quick questions about models in this library.

1. Could we know if there are any models that are good for text classification and summarisation tasks?
2. Could we know if those models can support running locally on CPU for inference?

Thanks!

[wanoz]

Hi thank you for the question, sorry for the late response. For text classification and summarisation, I have personally found value in the Google FLAN T5 models, I know the smaller versions of these models can be run on CPU for inference (such as base, small sizes). The performance of this will vary depending on your text data.