evaluation implementation

[lizhuorong]

Would you please release your evaluation implementation? Thank you and I am looking forward to that!

[lizhuorong]

Do you mean that the results in Table 1 of your paper is achieved by means of tests deployed on the AMT platform rather than the automatic measures?

[CQFIO]

The evaluation is essentially user study on Amazon Turk. I will update the evaluation code soon. It just creates javascript for Amazon Turk. There is no automatic measurement.

[tesfaldet]

+1 would like to see the mturk eval code! I'm currently working on a project where your mturk tests would serve as a great starting point.

[CQFIO]

Hi, I just upload the Amazon Turk scripts on Github. See folder "mturk_scripts".

[lizhuorong]

Thanks : )

[tesfaldet]

Awesome! Thanks!

[tesfaldet]

For the sake of completeness, mind sharing any relevant javascript code you use for the mturk experiments? Thank you!

[CQFIO]

@tesfaldet I forgot to put the Javascript code on GitHub. I just upload example code in "mturk_scripts".

[tesfaldet]

Thanks a lot for the quick responses :)

[tesfaldet]

Just a heads up that there's no javascript in unlimited_time_comparison/example_javascript.txt Not a problem for me though since I took it from limited_time_comparison/example_javascript.txt.