nmt icon indicating copy to clipboard operation
nmt copied to clipboard

why vocabs are the same between src and tgt in script "wmt16_en_de.sh"

Open kingkf opened this issue 7 years ago • 0 comments

Create vocabulary file for BPE

echo -e "\n\n" > "${OUTPUT_DIR}/vocab.bpe.${merge_ops}" cat "${OUTPUT_DIR}/train.tok.clean.bpe.${merge_ops}.en" "${OUTPUT_DIR}/train.tok.clean.bpe.${merge_ops}.de" |
${OUTPUT_DIR}/subword-nmt/get_vocab.py | cut -f1 -d ' ' >> "${OUTPUT_DIR}/vocab.bpe.${merge_ops}"

done

Duplicate vocab file with language suffix

cp "${OUTPUT_DIR}/vocab.bpe.32000" "${OUTPUT_DIR}/vocab.bpe.32000.en" cp "${OUTPUT_DIR}/vocab.bpe.32000" "${OUTPUT_DIR}/vocab.bpe.32000.de"

kingkf avatar Dec 22 '18 08:12 kingkf