multeval
multeval copied to clipboard
jhclark
Easy Bootstrap Resampling and Approximate Randomization for BLEU, METEOR, and TER using Multiple Optimizer Runs. This implements "Better Hypothesis Testing for Statistical Machine Translation: Control...
./multeval.sh eval --refs example/refs.test2010.lc.tok.en.* \ --baseline example/hyps.lc.tok.en.baseline.opt* \ --meteor.language en I get the following error while trying to run the sample files- Found existing METEOR installation at ./lib/meteor-1.4 Error: Could...
I'm having issues running multeval with METEOR in Arabic `--meteor.language ar`. Running independently (e.g. METEOR standalone) is no problem, but running it inside multeval gives: ``` RESULT: baseline: METEOR: RESAMPLED_MEAN_AVG:...
Hi, I typically run multeval for bleu and ter but haven't assessed statistical significance so far. Now that I actually need it , I find it (1) difficult to grasp...
When i am trying to calculate BLEU score using multi-eval. It is throwing me a error: Expected 999 references, but got 500 at multeval.HypothesisManager.loadRefs(HypothesisManager.java:66) at multeval.HypothesisManager.loadData(HypothesisManager.java:50) at multeval.MultEvalModule.run(MultEvalModule.java:109) at multeval.MultEval.main(MultEval.java:115)
>./multeval.sh eval --refs example/refs.test2010.lc.tok.en.* \ --hyps-baseline example/hyps.lc.tok.en.baseline.opt* \ --meteor.language en what it is 'example/refs.test2010.lc.tok.en.*' and 'hyps.lc.tok.en.baseline.opt*'. 我的理解是，--refs 后面跟的是原始的标注数据 --hyps-baseline 后面跟的是baseline系统跑出来的结果 我不能理解，为什么 --hyps-baseline后面跟的参数，在example里面是三个文件。
It seems TER is by default computed in case-insensitive mode and there is no way to pass the option to the TER scorer?
The latest version of multeval tries to read a file called "./constants" to determine its version number. When multeval is executed from a location other than its own folder, this...
When executing `./multeval.sh eval -R example/refs.test2010.lc.tok.en.0 --hyps-baseline example/hyps.lc.tok.en.baseline.opt0 --metrics ter` I get the following error: ``` Failed to specify required options: [refs] ``` But when changing `-R` to `--refs` everything...
It's hard to figure out how the build process works if someone clones this repository. The instructions are currently at the bottom of README.md. This change will move this to...
This fixes a regression introduced in commit [cb6a52](https://github.com/jhclark/multeval/commit/cb6a523f476dafede2a9ff8963638dcdde4f20b3), following a code example on [rgagnon.com](http://www.rgagnon.com/javadetails/java-0532.html), as mentioned on issue #9. It looks like `MultEval.loadVersion()` is only ever called from `MultEval.main()`, so...
