replicability-analysis-NLP
replicability-analysis-NLP copied to clipboard
Fisher Test unusual result
Hi, when I input the following P-Values:
[0.04, 0.1, 0.1, 0.07, 0.07, 0.07, 0.04]
With the following alpha level:
0.05
I get the following output:
{'dataset1': 0.04, 'dataset2': 0.1, 'dataset3': 0.1, 'dataset4': 0.07, 'dataset5': 0.07, 'dataset6': 0.07, 'dataset7': 0.04}
The K-Bonferroni estimator for the number of datasets with effect is: 0
The K-Fisher estimator for the number of datasets with effect is: 5
The output for K-Bonferroni is expected.
The output for K-Fisher I thought was surprising as the number of p-values greater than alpha is 5 of which if the p-value is greater than alpha there should be no effect from what I understand. I was wondering if the algorithm is working correctly? as in is this output what is expected?