torchchat
torchchat copied to clipboard
pytorch
Crashes with internal assert while parsing options
🐛 Describe the bug
See example below:
$ python3 torchchat.py generate --help
Traceback (most recent call last):
File "/home/nshulga/git/pytorch/torchchat/torchchat.py", line 53, in <module>
args = parser.parse_args()
^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 1869, in parse_args
args, argv = self.parse_known_args(args, namespace)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 1902, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 2096, in _parse_known_args
positionals_end_index = consume_positionals(start_index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 2073, in consume_positionals
take_action(action, args)
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 1978, in take_action
action(self, namespace, argument_values, option_string)
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 1241, in __call__
subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 1902, in parse_known_args
namespace, args = self._parse_known_args(args, namespace)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 2114, in _parse_known_args
start_index = consume_optional(start_index)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 2054, in consume_optional
take_action(action, args, option_string)
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 1978, in take_action
action(self, namespace, argument_values, option_string)
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 1119, in __call__
parser.print_help()
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 2601, in print_help
self._print_message(self.format_help(), file)
^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 2585, in format_help
return formatter.format_help()
^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 286, in format_help
help = self._root_section.format_help()
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 217, in format_help
item_help = join([func(*args) for func, args in self.items])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 217, in <listcomp>
item_help = join([func(*args) for func, args in self.items])
^^^^^^^^^^^
File "/home/nshulga/miniconda3/envs/py311/lib/python3.11/argparse.py", line 341, in _format_usage
assert ' '.join(opt_parts) == opt_usage
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError
Versions
Not sure
@Jack-Khuu , @byjlw are you on Linux or Mac? And what Python version are you using? I.e. same commit passes for me with py3.12 on Mac, but fails with py3.11 on Linux:
% git rev-parse HEAD; uname -a; python3 --version; python3 torchchat.py generate --help
b2c3f2697e2bbcda1b51f0c85adfaba27b900857
Darwin nshulga-mbp 23.5.0 Darwin Kernel Version 23.5.0: Wed May 1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64
Python 3.12.3
usage: torchchat generate [-h] [--checkpoint-path CHECKPOINT_PATH] [--dso-path DSO_PATH | --pte-path PTE_PATH] [--compile] [--compile-prefill] [--dtype {fp32,fp16,bf16,float,half,float32,float16,bfloat16,fast,fast16}] [--quantize QUANTIZE]
[--device {fast,cpu,cuda,mps}] [--prompt PROMPT] [--num-samples NUM_SAMPLES] [--max-new-tokens MAX_NEW_TOKENS] [--top-k TOP_K] [--temperature TEMPERATURE] [--hf-token HF_TOKEN] [--model-directory MODEL_DIRECTORY] [-v]
[--seed SEED]
[model]
vs
$ git rev-parse HEAD; uname -a; python3 --version; python3 torchchat.py generate --help
b2c3f2697e2bbcda1b51f0c85adfaba27b900857
Linux ip-10-200-84-248 5.15.0-1019-aws #23~20.04.1-Ubuntu SMP Thu Aug 18 03:20:14 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux
Python 3.11.5
Traceback (most recent call last):
File "/home/nshulga/git/pytorch/torchchat/torchchat.py", line 53, in <module>
args = parser.parse_args()
^^^^^^^^^^^^^^^^^^^
Was able to repro crash on Linux devGPU
CLI works on mac
b2c3f2697e2bbcda1b51f0c85adfaba27b900857
Darwin jackkhuu-mbp 23.5.0 Darwin Kernel Version 23.5.0: Wed May 1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64
Python 3.12.3
https://github.com/pytorch/torchchat/commit/4f937ac55746aa7f866dc0e827492c9a4a686d10
Was the first PR where it starts failing
https://stackoverflow.com/questions/60565750/python-argparse-assertionerror-when-using-mutually-exclusive-group
Seems like there might be some versioning difference when using the devGPU vs mac. The issue is a argparse library bug that has since been patched