Prompt details about evaluating on EditBench

[CharlesGong12]

Hi, what an amazing work! I found that you evaluated BrushNet on editbench in your paper. In the editbench's annotation file, there are several kinds of prompts: prompt_full, prompt_mask-simple, prompt_mask-rich and the like. So which prompt did you use when evaluating?

[CharlesGong12]

And there seems no refined EditBench in the link you provide. Looking forward to your reply!

[juxuan27]

Hi, @CharlesGong12 ! I use prompt_full for evaluation. The refined EditBench directly uses the mask of EditBench (by extracting the black part). We can not release the refined EditBench due to license issues, but it is really easy to process. Sorry for the trouble!

[CharlesGong12]

Thanks for your reply! I am confused why not use prompt_mask, the local description for inpainting area? It seems that using local description is much more common in real inpainting scenarios, such as replacing a red shirt with a blue shirt and using prompt "blue shirt". @juxuan27

[zhLawliet]

+1