graspologic icon indicating copy to clipboard operation
graspologic copied to clipboard

Allow fast DCorr for LatentDistributionTest

Open bdpedigo opened this issue 5 years ago • 3 comments

Looks like we turned off the ability to use fast chi-squared approximation for DCorr LatentDistributionTest via the auto kwarg in this line.

Any reason we want to omit this behavior?

bdpedigo avatar Sep 21 '20 22:09 bdpedigo

question for @alyakin314 though you don't necessarily need to fix if we end up wanting to support this

bdpedigo avatar Sep 21 '20 22:09 bdpedigo

@bdpedigo, in short: we (as in you and I) had no idea of whether it leads to a valid / consistent test in the graph setting, and whether is it more or less conservative than the permutation test.

the proof of the convergence of the distribution to chi-squared obviously assumes that the samples are independent, but we know that in the graph setting they aren't. this is actually one of the assumptions of the permutation test as well, by breaking which we end up with a test that still works, but is overly conservative (this is why our practical size is less than alpha, even though under all assumptions of the permutation tests satisfied, it should be exactly alpha). we knew the behavior of the permutation test, but had no idea how fast dcorr will work with graphs, so we decided to bother about it later. i think there is discussion between you and me about this in the hyppo->graspy pr.

if you ask me - my intuition tells me that using fast dcorr + graphs will work, and the test will be more conservative than it should according to fast dcorr theory. whether it will be more or less conservative than the permutation test - me not knows, but probably more if anything. this is something that should be investigated prior to adding the capability of flipping that flag, which i fully support.

alyakin314 avatar Sep 22 '20 11:09 alyakin314

Ya I remember discussing now, thanks for the explanation!

bdpedigo avatar Sep 22 '20 11:09 bdpedigo