Mikko Aulamo
Results
2
comments of
Mikko Aulamo
There is now `--chunk_size` parameter to control memory consumption, although the current implementation is still slow for corpora with huge documents. Regarding moses, it also possible to download moses files...
You are probably using tokenized preprocessing which is the default. You can produce untokenized output with the `-p raw` option.