insight-bench Data/Annotation Issues?

notebook 12 annotation seems very off. The observed insights seem to in no way relate to what the analysis actually shows. All incidents have equal category distribution but the insight says "The number of Hardware incidents is significantly higher than others". There are no frequency of printers found but the insight says "Most of the hardware incidents are related to printer issues".

May 18 '25 08:05 george1459

And, to a slightly less extent but another quite serious issue: in notebook 8, I am not sure how the first 2 insights are drawn. The number of incidents assigned are the same to everyone. The same issue appears in insight 2 in notebook 9

May 18 '25 08:05 george1459

Thanks for flagging this @george1459. Yes, there seems to be some versioning issues in the some of the initial notebooks (some of them are partially correct, some aren't). We are working to resolve this issue. Again, so sorry for the late reply. We greatly appreciate your interest in our benchmark.

Jun 02 '25 00:06 Demfier

Hey @george1459

Thanks for your patience, we have now fixed the discrepancy between the graphs and their annotations. Apologies for the delay again.

Feel free to reach out again in case you have any questions.

Jul 17 '25 18:07 Demfier