Problem with unrecognized argument

[sushantkumar007007]

When I run the CodeGen, the preprocessing.py it is asking for various arguments and when I pass those argument its showing unrecognized arguments. Below are my codes. %run codegen_sources/preprocessing/preprocess data/test_dataset 20 python obfuscation 8 500 200 400 roberta False data/bpe/cpp-java-python/vocab False data/bpe/cpp-java-python --1

usage: preprocess.py [-h] [--local LOCAL] [--local_parallelism LOCAL_PARALLELISM] [--langs LANGS [LANGS ...]] [--mode {obfuscation,monolingual,monolingual_functions,obfuscation_functions}] [--train_splits TRAIN_SPLITS] [--job_mem JOB_MEM] [--tokenization_timeout TOKENIZATION_TIMEOUT] [--bpe_timeout BPE_TIMEOUT] [--train_bpe_timeout TRAIN_BPE_TIMEOUT] [--bpe_mode {fast,roberta}] [--fastbpe_use_vocab FASTBPE_USE_VOCAB] [--fastbpe_vocab_path FASTBPE_VOCAB_PATH] [--keep_comments KEEP_COMMENTS] [--fastbpe_code_path FASTBPE_CODE_PATH] [--ncodes NCODES] [--percent_test_valid PERCENT_TEST_VALID] input_path preprocess.py: error: unrecognized arguments: 20 python obfuscation 8 500 200 400 roberta False data/bpe/cpp-java-python/vocab False data/bpe/cpp-java-python --1

[baptisteroziere]

Hi, Is this line the command you run in a terminal?

%run codegen_sources/preprocessing/preprocess data/test_dataset 20 python obfuscation 8 500 200 400 roberta False data/bpe/cpp-java-python/vocab False data/bpe/cpp-java-python --1

I believe that you are not passing the arguments correctly. Maybe your script omits the -- before named arguments (e.g. --local_parallelism 20)