HF_ENDPOINT=https://hf-mirror.com python eval.py --model "hf" --tasks "flare_ner,flare_sm_acl,flare_fpb" --model_args "pretrained=/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/2_select_merge,tokenizer=/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/2_select_merge"
[nltk_data] Downloading package punkt to /home/jovyan/nltk_data...
[nltk_data] Package punkt is already up-to-date!
Selected Tasks: ['flare_fpb', 'flare_ner', 'flare_sm_acl']
Using device 'cuda'
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 2.34it/s]
Using custom data configuration chancefocus--flare-fpb-410c0d321cd6b563
Downloading and preparing dataset parquet/chancefocus--flare-fpb to /home/jovyan/.cache/huggingface/datasets/parquet/chancefocus--flare-fpb-410c0d321cd6b563/0.0.0/0b6d5799bb726b24ad7fc7be720c170d8e497f575d02d47537de9a5bac074901...
Downloading data files: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 5021.11it/s]
Extracting data files: 100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 379.23it/s]
Traceback (most recent call last):
File "/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/PIXIU/src/eval.py", line 97, in
main()
File "/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/PIXIU/src/eval.py", line 62, in main
results = evaluator.simple_evaluate(
File "/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/PIXIU/src/financial-evaluation/lm_eval/utils.py", line 243, in _wrapper
return fn(*args, **kwargs)
File "/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/PIXIU/src/evaluator.py", line 97, in simple_evaluate
task_dict = ta.get_task_dict(tasks)
File "/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/PIXIU/src/tasks/init.py", line 120, in get_task_dict
task_name_dict = {
File "/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/PIXIU/src/tasks/init.py", line 121, in
task_name: get_task(task_name)()
File "/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/PIXIU/src/financial-evaluation/lm_eval/base.py", line 510, in init
self.download(data_dir, cache_dir, download_mode)
File "/home/jovyan/jhy/LLaMA-Factory/_jhy/finance/PIXIU/src/financial-evaluation/lm_eval/base.py", line 539, in download
self.dataset = datasets.load_dataset(
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/load.py", line 1687, in load_dataset
builder_instance.download_and_prepare(
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/builder.py", line 605, in download_and_prepare
self._download_and_prepare(
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/builder.py", line 694, in _download_and_prepare
self._prepare_split(split_generator, **prepare_split_kwargs)
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/builder.py", line 1154, in _prepare_split
writer.write_table(table)
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/arrow_writer.py", line 507, in write_table
self._build_writer(inferred_schema=pa_table.schema)
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/arrow_writer.py", line 341, in _build_writer
inferred_features = Features.from_arrow_schema(inferred_schema)
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/features/features.py", line 1250, in from_arrow_schema
return Features.from_dict(metadata["info"]["features"])
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/features/features.py", line 1279, in from_dict
obj = generate_from_dict(dic)
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/features/features.py", line 1073, in generate_from_dict
return {key: generate_from_dict(value) for key, value in obj.items()}
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/features/features.py", line 1073, in
return {key: generate_from_dict(value) for key, value in obj.items()}
File "/home/jovyan/.conda/envs/jhy_pixiu/lib/python3.10/site-packages/datasets/features/features.py", line 1077, in generate_from_dict
return Sequence(feature=generate_from_dict(obj["feature"]), length=obj["length"])
KeyError: 'length'
The-FinAI
