Oscar icon indicating copy to clipboard operation
Oscar copied to clipboard

Clarification of results

Open alfred100p opened this issue 4 years ago • 3 comments

Hi in both the table in the readme and paper you mention a column for image captioning as C could someone verify if it's CIDER or CIDER-D.

alfred100p avatar Aug 21 '21 05:08 alfred100p

The caption evaluation scores are calculated by the official COCO evaluation code, which is a submodule in this repo. You may check the evaluation code for details of CIDEr.

xiaoweihu avatar Aug 24 '21 01:08 xiaoweihu

Thank you so based on the last lines of:

  • Above mentioned code https://github.com/LuoweiZhou/coco-caption/blob/master/pycocoevalcap/cider/cider.py
  • Official CIDEr implementation https://github.com/vrama91/cider/blob/master/pyciderevalcap/cider/cider.py
  • Official CIDEr-D implementarion https://github.com/vrama91/cider/blob/master/pyciderevalcap/ciderD/ciderD.py

I assume it was CIDEr. But I assume it would be better if an official author could comment on what was intended in the paper.

alfred100p avatar Aug 24 '21 05:08 alfred100p

Based on the table 1 from the paper it mentions AoANet (citation 11) and in its official paper (https://arxiv.org/pdf/2002.12204.pdf) in table 1 the score can be concluded as CIDEr-D. I have made a PR to add the score names to the README file(#137).

alfred100p avatar Aug 24 '21 07:08 alfred100p