Source-:
Hugging-face documentation.
My code-:
My error log
| Title | Author | Genre | SubGenre | Height | Publisher
-- | -- | -- | -- | -- | -- | --
Fundamentals of Wavelets | Goswami, Jaideva | tech | signal_processing | 228 | Wiley
Data Smart | Foreman, John | tech | data_science | 235 | Wiley
God Created the Integers | Hawking, Stephen | tech | mathematics | 197 | Penguin
Superfreakonomics | Dubner, Stephen | science | economics | 179 | HarperCollins
Orientalism | Said, Edward | nonfiction | history | 197 | Penguin
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
Cell In[25], line 1
----> 1 model = AutoModelForCausalLM.from_pretrained("google/gemma-2b", quantization_config=quantization_config)
File m:\Third Year\Sixth Semester\Projects\RAG_project1\venv\Lib\site-packages\transformers\models\auto\auto_factory.py:561, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
559 elif type(config) in cls._model_mapping.keys():
560 model_class = _get_model_class(config, cls._model_mapping)
--> 561 return model_class.from_pretrained(
562 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
563 )
564 raise ValueError(
565 f"Unrecognized configuration class {config.__class__} for this kind of AutoModel: {cls.__name__}.\n"
566 f"Model type should be one of {', '.join(c.__name__ for c in cls._model_mapping.keys())}."
567 )
File m:\Third Year\Sixth Semester\Projects\RAG_project1\venv\Lib\site-packages\transformers\modeling_utils.py:3024, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
3021 hf_quantizer = None
3023 if hf_quantizer is not None:
-> 3024 hf_quantizer.validate_environment(
3025 torch_dtype=torch_dtype, from_tf=from_tf, from_flax=from_flax, device_map=device_map
3026 )
3027 torch_dtype = hf_quantizer.update_torch_dtype(torch_dtype)
3028 device_map = hf_quantizer.update_device_map(device_map)
...
69 "Converting into 4-bit or 8-bit weights from tf/flax weights is currently not supported, please make"
70 " sure the weights are in PyTorch format."
71 )
ImportError: Using `bitsandbytes` 8-bit quantization requires Accelerate: `pip install accelerate` and the latest version of bitsandbytes: `pip install -i https://pypi.org/simple/ bitsandbytes`
Title Author Genre SubGenre Height Publisher
0 Fundamentals of Wavelets Goswami, Jaideva tech signal_processing 228 Wiley
1 Data Smart Foreman, John tech data_science 235 Wiley
2 God Created the Integers Hawking, Stephen tech mathematics 197 Penguin
3 Superfreakonomics Dubner, Stephen science economics 179 HarperCollins
4 Orientalism Said, Edward nonfiction history 197 Penguin
Token is valid (permission: read).
[1m[31mCannot authenticate through git-credential as no helper is defined on your machine.
You might have to re-authenticate when pushing to the Hugging Face Hub.
Run the following command in your terminal in case you want to set the 'store' credential helper as default.
git config --global credential.helper store
Read https://git-scm.com/book/en/v2/Git-Tools-Credential-Storage for more details.[0m
Token has not been saved to git credential helper.
Your token has been saved to C:\Users\HP\.cache\huggingface\token
Login successful
---------------------------------------------------------------------------
ImportError Traceback (most recent call last)
Cell In[25], [line 1](vscode-notebook-cell:?execution_count=25&line=1)
----> [1](vscode-notebook-cell:?execution_count=25&line=1) model = AutoModelForCausalLM.from_pretrained("google/gemma-2b", quantization_config=quantization_config)
File m:\Third Year\Sixth Semester\Projects\RAG_project1\venv\Lib\site-packages\transformers\models\auto\auto_factory.py:561, in _BaseAutoModelClass.from_pretrained(cls, pretrained_model_name_or_path, *model_args, **kwargs)
559 elif type(config) in cls._model_mapping.keys():
560 model_class = _get_model_class(config, cls._model_mapping)
--> 561 return model_class.from_pretrained(
562 pretrained_model_name_or_path, *model_args, config=config, **hub_kwargs, **kwargs
563 )
564 raise ValueError(
565 f"Unrecognized configuration class {config.class} for this kind of AutoModel: {cls.name}.\n"
566 f"Model type should be one of {', '.join(c.name for c in cls._model_mapping.keys())}."
567 )
File m:\Third Year\Sixth Semester\Projects\RAG_project1\venv\Lib\site-packages\transformers\modeling_utils.py:3024, in PreTrainedModel.from_pretrained(cls, pretrained_model_name_or_path, config, cache_dir, ignore_mismatched_sizes, force_download, local_files_only, token, revision, use_safetensors, *model_args, **kwargs)
3021 hf_quantizer = None
3023 if hf_quantizer is not None:
-> 3024 hf_quantizer.validate_environment(
3025 torch_dtype=torch_dtype, from_tf=from_tf, from_flax=from_flax, device_map=device_map
3026 )
3027 torch_dtype = hf_quantizer.update_torch_dtype(torch_dtype)
3028 device_map = hf_quantizer.update_device_map(device_map)
...
69 "Converting into 4-bit or 8-bit weights from tf/flax weights is currently not supported, please make"
70 " sure the weights are in PyTorch format."
71 )
ImportError: Using bitsandbytes 8-bit quantization requires Accelerate: pip install accelerate and the latest version of bitsandbytes: pip install -i https://pypi.org/simple/ bitsandbytes
Note
I have already installed accelerate and bitsandbytes
But I still have one confusion the log say that for 8-bit quantisation I need accelerate and other package, but I am doing 4 bit quantization.
Note I have already pip install accelerate, bitsandbytes
https://stackoverflow.com/questions/76924239/accelerate-and-bitsandbytes-is-needed-to-install-but-i-did
this might have the solution to your problem
google-deepmind
