Ethan Chern

Hi @zhimin-z . If you are referring to the Self-Check baselines, then yes. Please refer to https://github.com/GAIR-NLP/factool/blob/main/factool/utils/prompts/self_check.yaml

Hi @zhimin-z. These results were evaluated neither using 0-shot nor 3-shot. The results were evaluated using FacTool. If you are referring to how the individual modules in FacTool are implemented,...

it might be related to this issue: https://github.com/GAIR-NLP/factool/issues/33

I see. maybe change `response_list = factool_instance.run(inputs)` to `response_list = asyncio.run(factool_instance.run(inputs))`. Remember to `import asyncio`. Also, for Scientific Literature Review, the scholarly API may have some bugs (please refer to...

I see. Just curious, does this issue happen in other tasks (KBQA, math, code)? It is possible that the response doesn't contain any valid paper so the claim extraction process...

Thanks for the feedback! So I think the current version of FacTool inherently assumes that the given response should include at least one claim (i.e., the response contains the name...

Hi @LeiyanGithub, thank you for your interest in our paper and for reaching out! You've understood correctly. We used the [dataset](https://github.com/GAIR-NLP/factool/tree/main/datasets) that we constructed for our experiments. The claims in...

Hi @LeiyanGithub, thanks for asking! I will upload our result and the metric evaluation code in the next few days. However, I want to apologize in advance, as I am...

@YaxinFAN1, thank you for your interest in our work!!! I just modified the [openai_wrapper](https://github.com/GAIR-NLP/factool/blob/main/factool/utils/openai_wrapper.py) to catch `openai.error.APIConnectionError`. If this doesn't work, the issue is likely still related to the proxy,...

@fire17 Thank you for your interest on our work! Indeed adding private knowledge support to FacTool is on our priority list! To achieve this, we have to change this file...