/code/tensorrt_llm# python examples/llama/convert_checkpoint.py --model_dir /code/tensorrt_llm/Mixtral-8x7B-Instruct-v0.1/ --dtype float16 --output_dir /code/tensorrt_llm/examples/Mixtral-8x7B-Instruct-v0.1/ --int8_kv_cache --dataset-cache-dir .
[TensorRT-LLM] TensorRT-LLM version: 0.10.0.dev2024041600
0.10.0.dev2024041600
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████| 19/19 [00:14<00:00, 1.35it/s]
[04/19/2024-05:28:34] Some parameters are on the meta device device because they were offloaded to the cpu.
/usr/local/lib/python3.10/dist-packages/datasets/load.py:1461: FutureWarning: The repository for ccdv/cnn_dailymail contains custom code which must be executed to correctly load the dataset. You can inspect the repository content at https://hf.co/datasets/ccdv/cnn_dailymail
You can avoid this message in future by passing the argument trust_remote_code=True.
Passing trust_remote_code=True will be mandatory to load this dataset from the next major release of datasets.
warnings.warn(
Downloading builder script: 100%|████████████████████████████████████████████████| 9.27k/9.27k [00:00<00:00, 38.4MB/s]
Downloading readme: 100%|████████████████████████████████████████████████████████| 13.9k/13.9k [00:00<00:00, 54.7MB/s]
Traceback (most recent call last):
File "/code/tensorrt_llm/examples/llama/convert_checkpoint.py", line 446, in
main()
File "/code/tensorrt_llm/examples/llama/convert_checkpoint.py", line 438, in main
convert_and_save_hf(args)
File "/code/tensorrt_llm/examples/llama/convert_checkpoint.py", line 345, in convert_and_save_hf
LLaMAForCausalLM.quantize(args.model_dir,
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/llama/model.py", line 363, in quantize
convert.quantize(
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/llama/convert.py", line 1244, in quantize
act_range, llama_qkv_para, llama_smoother = smooth_quant(
File "/usr/local/lib/python3.10/dist-packages/tensorrt_llm/models/llama/convert.py", line 1037, in smooth_quant
dataset = load_dataset("ccdv/cnn_dailymail",
File "/usr/local/lib/python3.10/dist-packages/datasets/load.py", line 2582, in load_dataset
builder_instance.download_and_prepare(
File "/usr/local/lib/python3.10/dist-packages/datasets/builder.py", line 1005, in download_and_prepare
self._download_and_prepare(
File "/usr/local/lib/python3.10/dist-packages/datasets/builder.py", line 1767, in _download_and_prepare
super()._download_and_prepare(
File "/usr/local/lib/python3.10/dist-packages/datasets/builder.py", line 1078, in download_and_prepare
split_generators = self.split_generators(dl_manager, *split_generators_kwargs)
File "/root/.cache/huggingface/modules/datasets_modules/datasets/ccdv--cnn_dailymail/0107f7388b5c6fae455a5661bcd134fc22da53ea75852027040d8d1e997f101f/cnn_dailymail.py", line 255, in _split_generators
dl_paths = dl_manager.download_and_extract(_DL_URLS)
File "/usr/local/lib/python3.10/dist-packages/datasets/download/download_manager.py", line 570, in download_and_extract
return self.extract(self.download(url_or_urls))
File "/usr/local/lib/python3.10/dist-packages/datasets/download/download_manager.py", line 434, in download
downloaded_path_or_paths = map_nested(
File "/usr/local/lib/python3.10/dist-packages/datasets/utils/py_utils.py", line 475, in map_nested
mapped = [
File "/usr/local/lib/python3.10/dist-packages/datasets/utils/py_utils.py", line 476, in
_single_map_nested((function, obj, types, None, True, None))
File "/usr/local/lib/python3.10/dist-packages/datasets/utils/py_utils.py", line 368, in _single_map_nested
return function(data_struct)
File "/usr/local/lib/python3.10/dist-packages/datasets/download/download_manager.py", line 459, in _download
out = cached_path(url_or_filename, download_config=download_config)
File "/usr/local/lib/python3.10/dist-packages/datasets/utils/file_utils.py", line 190, in cached_path
output_path = get_from_cache(
File "/usr/local/lib/python3.10/dist-packages/datasets/utils/file_utils.py", line 589, in get_from_cache
raise ConnectionError(f"Couldn't reach {url} ({repr(head_error)})")
ConnectionError: Couldn't reach https://huggingface.co/datasets/ccdv/cnn_dailymail/resolve/main/cnn_stories.tgz (ConnectionError(MaxRetryError("HTTPSConnectionPool(host='cdn-lfs.huggingface.co', port=443): Max retries exceeded with url: /datasets/ccdv/cnn_dailymail/e8fbc0027e54e0a916abd9c969eb35f708ed1467d7ef4e3b17a56739d65cb200?response-content-disposition=attachment%3B+filename%3DUTF-8%27%27cnn_stories.tgz%3B+filename%3D%22cnn_stories.tgz%22%3B&Expires=1713763720&Policy=eyJTdGF0ZW1lbnQiOlt7IkNvbmRpdGlvbiI6eyJEYXRlTGVzc1RoYW4iOnsiQVdTOkVwb2NoVGltZSI6MTcxMzc2MzcyMH19LCJSZXNvdXJjZSI6Imh0dHBzOi8vY2RuLWxmcy5odWdnaW5nZmFjZS5jby9kYXRhc2V0cy9jY2R2L2Nubl9kYWlseW1haWwvZThmYmMwMDI3ZTU0ZTBhOTE2YWJkOWM5NjllYjM1ZjcwOGVkMTQ2N2Q3ZWY0ZTNiMTdhNTY3MzlkNjVjYjIwMD9yZXNwb25zZS1jb250ZW50LWRpc3Bvc2l0aW9uPSoifV19&Signature=pZ3ot5krXIXBDiyZR6c8oaijndL0nxBmrqC7mGxL25Fj8-lX5~h14QDfJTO1eI~tvdkTbzxU7bc7eP0HbQ0cP8FRxcZGVwavW5hIj~h-xyy~5rsywjmsudTDJwNpk103W1aHBPGM03FVXAGNPTYxa~e4fIPdYDRTREFuGCoQlMNLc4PlsRh0bvUKe1w1bsc9hmvOaD36xJoP-1jwHiL16OE1WbaB7qte-rBsUktXeMi1hNS~darFES46Das1vwqrlGl7b5IRyu5OIF6-t~dyMvufCFR4e0yUcNW7e3kuEzURiwOkT91N7evOu8DQkrNdagJgMuS078YvkxQZtgeZjQ&Key-Pair-Id=KVTP0A1DKRTAX (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7359eb95c520>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))")))
Its not able to Download the https://huggingface.co/datasets/ccdv/cnn_dailymail/resolve/main/cnn_stories.tgz file from within the docker.
It looks an issue of the dataset. We could not bring too many helps. You could try download the datasets manually and prevent downloading in the program.
NVIDIA
