panml

panml copied to clipboard

Models for text classification & summarisation, and running on CPU

1

Hi, Many thanks for releasing this library for using LLMs! We just have two quick questions about models in this library. 1. Could we know if there are any models...

Dee-Ma

Model size and memory needed to run

2

Is there a way to figure in **advance** how much memory : 1. It will require to LOAD the model 2. It will require to RUN the model f.e. when...

vsraptor

tiiuae/falcon-7b-instruct : ValueError: The following `model_kwargs` are not used by the model: ['token_type_ids']

1

``` In [7]: x=LLM() tiiuae/falcon-7b-instruct Explicitly passing a `revision` is encouraged when loading a configuration with custom code to ensure no malicious code has been contributed in a newer revision....

vsraptor

Plan for adding support for additional models like Llama

1

Just curious if you've a plan in place/timeline to expand support for other models from the HuggingFace hub

satreja1

Issue loading lora model

2

Trained flan model using lora technique but while loading it from local, gives the following errors: ```HFValidationError: Repo id must be in the form 'repo_name' or 'namespace/repo_name': './results/model_flan_lora'. Use repo_type...

Aaryanverma

Suggestion: Add more examples for different models like Bloomz, Dolly, GPT etc.

1

Add more examples for different models like Bloomz, Dolly, GPT etc.

Aaryanverma

Idea: add option to load models with 8bit and 4bit quantization

1

Less memory requirement for 4bit and 8bit quantized models. https://huggingface.co/docs/transformers/main/main_classes/quantization

Hanshan1988

wanoz

Idea: look into DPO for model tuning

1

wanoz

wanoz