netZooPy icon indicating copy to clipboard operation
netZooPy copied to clipboard

Singular shrunken covariance matrix in DRAGON

Open katehoffshutta opened this issue 2 years ago • 3 comments

The DRAGON precision matrix is the inverse of the shrunken covariance (1-lambda)S + lambdaT where T is the target matrix of covariances on the diagonal and should be full rank. However, we do not currently have a check to see if T is full rank. If any of the variables have zero variance, (1-lambda)S + lambdaT can be singular. DRAGON will not recognize this problem; it will try to invert the matrix and the numpy.linalg matrix inversion function will throw an exception that is unclear.

I plan to handle this by adding default behavior that any variables with zero variance are excluded from the analysis and a note warning of the excluded variables is printed to the console.

I also will add a try-catch on the rank of (1-lambda)S + lambdaT so that DRAGON provides an informative exception before it can percolate down to numpy.linalg.

katehoffshutta avatar Apr 21 '23 13:04 katehoffshutta

Note that this can also occur in LIONESS-DRAGON. I will implement similar fixes here.

katehoffshutta avatar May 26 '23 01:05 katehoffshutta

Ehi Kate, is this still something you will work on in the future? I assigned it both a bug and an enhancement label for now

violafanfani avatar Sep 27 '23 13:09 violafanfani

Thanks, Viola - this is accurate. We will try to get this in for the next release.

katehoffshutta avatar Sep 28 '23 14:09 katehoffshutta