raid icon indicating copy to clipboard operation
raid copied to clipboard

ai-text-detector-v-45.5

Open kishankachhadiya opened this issue 3 months ago • 5 comments

@liamdugan Hello! We want to evaluate our model, thanks!!!

kishankachhadiya avatar Nov 04 '25 09:11 kishankachhadiya

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

ai-text-detector-v-45.5

Release date: 2025-11-04

I've committed detailed results of this detector's performance on the test set to this PR.

[!WARNING] Failed to find threshold values that achieve False Positive Rate(s): (['5%', '1%']) on all domains. This submission will not appear in the main leaderboard for those FPR values; it will only be visible within the splits in which the target FPR was achieved.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

github-actions[bot] avatar Nov 05 '25 01:11 github-actions[bot]

@liamdugan Hello! I’ve retrained my model, debarta-text-classifier, and would like to request an evaluation of its performance. Could you please review the updated model and provide feedback on its results?

kishankachhadiya avatar Nov 07 '25 08:11 kishankachhadiya

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

debarta-text-classifier

Release date: 2025-11-06

I've committed detailed results of this detector's performance on the test set to this PR.

[!WARNING] Failed to find threshold values that achieve False Positive Rate(s): (['1%']) on all domains. This submission will not appear in the main leaderboard for those FPR values; it will only be visible within the splits in which the target FPR was achieved. On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an AUROC of 82.42 and a TPR of 65.32% at FPR=5%. Without adversarial attacks, it achieved AUROC of 92.60 and a TPR of 81.25% at FPR=5%.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

github-actions[bot] avatar Nov 07 '25 17:11 github-actions[bot]

@liamdugan I have retrained my model. Please evaluate my latest model.

kishankachhadiya avatar Nov 13 '25 04:11 kishankachhadiya

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

debarta-text-classifier-v1

Release date: 2025-11-10

I've committed detailed results of this detector's performance on the test set to this PR.

[!WARNING] Failed to find threshold values that achieve False Positive Rate(s): (['1%']) on all domains. This submission will not appear in the main leaderboard for those FPR values; it will only be visible within the splits in which the target FPR was achieved. On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an AUROC of 98.36 and a TPR of 94.94% at FPR=5%. Without adversarial attacks, it achieved AUROC of 98.84 and a TPR of 97.98% at FPR=5%.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

github-actions[bot] avatar Nov 18 '25 06:11 github-actions[bot]