tylin
How to contribute
Hi, Thank you very much for the evaluation script. For my applications, I needed a Python 3.5 version of this script so I converted this code to be compatible with Python 3.5 and was wondering whether I can create a pull request to include it either in this repo or somewhere else.
Thanks again
Hey there, I'm also interested in this.
How did you deal with byte vs unicode strings? METEOR crashes at this line and if I change it to self.meteor_p.stdin.write(str.encode('{}\n'.format(score_line))) the process just hangs there indefinitely.
I've tried it on both python 3.5 and 3.6.
@PavlosMelissinos i have come across the same problem about meteor when importing it into python3, have you solved your problem (all other metrics works properly)?
No, for the foreseeable future I plan to ignore the metric. CIDEr (and SPICE, which you can find on this fork) are the ones that matter the most for image captioning anyway.
If you're interested in alternative metrics, SPIDEr is a combination of the two that allegedly works better but I'm not aware of any implementations, I'd try to do it myself but unfortunately I don't have enough free time right now.
Word Mover's Distance is another metric that seems promising that works on the word-vector space but I haven't tested it in practice.
Hey! I am interested in the Python 3.5 version of the script too.
@wgharbieh what happened with the pull request? Can we find the code in a repository?
