how to split single-triplet and multi-triplet

[suolyer]

hi~, nice work I have some question. how to get the single-triplet dataset and multi-triplet dataset？ Why is the accuracy rate used for single-triplet instead of F1? The model may obtain multiple triplets during prediction, or a single triplet cannot be generated. How to solve this situation?

[chiayewken]

Hi, for the single-triplet case, the model is restricted to one triplet prediction, hence we specifically use the accuracy metric which is equivalent to F1 in this case.

For the multi-triplet case, we do not have any constraints on the model prediction, hence the F1 metric is used which is consistent with other information extraction tasks.

For the single and multi-triplet datasets, the data processing steps for evaluation can be found here: https://github.com/declare-lab/RelationPrompt/blob/ab415648e05d2feb8ba1e94d7726f2032273d14e/wrapper.py#L438