Jinjie Ni

can you replicate the results in paper? I ran the same recipe as yours, got a test ppl of 29.14, but the results in paper should be 18.7.

@troyas ValueError: Cannot register duplicate model (convtransformer_simul_trans) This error happens when you register a model that has already existed in the fairseq pkg. Try to change your register name at...

> Hi @Psycoy, sorry for the late reply. Mine was 19.7, and despite being close to 18.7, it still has a gap. Can you reproduce their results? Yes, I can,...

可能是由于windows的路径和linux不一致导致的。我刚才修复了有可能导致该错误的bug。 目前我手边没有windows系统的机器，麻烦你重新安装并测试一下： `pip install --upgrade easyliter` 之前的版本只在mac和linux测试过。 注意：不要使用相对路径，尽量使用绝对路径。

又更新了下。这个版本还不行的话先用绝对路径，我改天用windows虚拟机测试下。

这是正常的，google scholar有时候会墙一些活动频繁的用户。下载链接主要依赖google scholar和arxiv的解析，如果没有的话就需要依靠dlbp, sci-hub等一些其他相对较小的数据库。如果这些数据库都找不到pdf链接，那就需要自己手动下载一下pdf放入pdf文件夹，然后在markdown填入文件名即可（markdown文件里面已经把其他所有格式准备好了，只需要在`[pdf]`的的地方填入pdf文件名即可）。

Hi @wildphoton , we are working on the local support of this feature (insufficient bandwidth recently lol). But if you want, you could refer to [here](https://github.com/Psycoy/MixEval/blob/main/docs/evaluate_instructions.md#use-other-apis-for-model-parser) to use the api...

Hi @wildphoton , We have integrated the OSS local judge feature. You can now use any language model on the huggingface to judge. You can configure the `OSJudgeCloseendFreeform` and `OSJudgeCloseendMultichoice`...

Hi, Waneila, you can run the test on MixEval-Hard and MixEval and give us the screenshot here Please make sure to adhere to the instructions in the repo thanks Jinjie

Hi @Waneila , I cannot see the results here, was it attached as an image?