HiCExplorer icon indicating copy to clipboard operation
HiCExplorer copied to clipboard

hicCorrelate issue

Open raquelsofi opened this issue 3 years ago • 7 comments

Hi,

I'm trying to run hicCorrelate on my set of 4 raw matrices with a bin size of 500 kb, and keep getting an error "arrays used as indices must be of integer (or boolean) type" as below. I've also tried to play around with using the range function but this gave the same error. Also the same if i use Spearman correlation. Any suggestions on how i could fix this? Thank you in advance!

hicInfo 3.0 Python 3.9.7 -used conda to install latest version

~/HiCExplorer_output_test $ hicCorrelate -m h5format/Sample_S39783_E12-BCRpos_raw_500000.h5 h5format/Sample_S39784_F12-BCRpos_raw_500000.h5 h5format/Sample_S39785_G12-BCRneg_raw_500000.h5 h5format/Sample_S39786_H12-BCRneg_raw_500000.h5 \

--method=pearson
--log1p
--labels E12-BCRpos F12-BCRpos G12-BCRneg H12-BCRneg
--outFileNameHeatmap hicCorrelate_pearson_raw_500000_heatmap
--outFileNameScatter hicCorrelate_pearson_raw_500000_scatterplot
--plotFileFormat png INFO:numexpr.utils:Note: detected 96 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable. INFO:numexpr.utils:Note: NumExpr detected 96 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. INFO:numexpr.utils:NumExpr defaulting to 8 threads. INFO:hicexplorer.hicCorrelate:loading hic matrix h5format/Sample_S39783_E12-BCRpos_raw_500000.h5

INFO:hicexplorer.hicCorrelate:restore masked bins h5format/Sample_S39783_E12-BCRpos_raw_500000.h5

INFO:hicexplorer.hicCorrelate:loading hic matrix h5format/Sample_S39784_F12-BCRpos_raw_500000.h5

INFO:hicexplorer.hicCorrelate:restore masked bins h5format/Sample_S39784_F12-BCRpos_raw_500000.h5

INFO:hicexplorer.hicCorrelate:loading hic matrix h5format/Sample_S39785_G12-BCRneg_raw_500000.h5

INFO:hicexplorer.hicCorrelate:restore masked bins h5format/Sample_S39785_G12-BCRneg_raw_500000.h5

INFO:hicexplorer.hicCorrelate:loading hic matrix h5format/Sample_S39786_H12-BCRneg_raw_500000.h5

INFO:hicexplorer.hicCorrelate:restore masked bins h5format/Sample_S39786_H12-BCRneg_raw_500000.h5

Traceback (most recent call last): File "/storage/home/rsilva/anaconda3/bin/hicCorrelate", line 7, in main() File "/storage/home/rsilva/anaconda3/lib/python3.9/site-packages/hicexplorer/hicCorrelate.py", line 328, in main rows_keep = cols_keep = np.delete(list(range(all_mat.shape[1])), all_nan) File "<array_function internals>", line 5, in delete File "/storage/home/rsilva/anaconda3/lib/python3.9/site-packages/numpy/lib/function_base.py", line 4480, in delete keep[obj,] = False IndexError: arrays used as indices must be of integer (or boolean) type

raquelsofi avatar Feb 25 '22 10:02 raquelsofi

Can you check if your matrices contain data? Do they contain any problematic regions like scaffolds or contigs? Errors like this usually appear if a sub-arrray selection is done via index / or Boolean values, but they contain no data. The usual reason is area in the matrix without any data.

joachimwolff avatar Feb 28 '22 17:02 joachimwolff

I have occured same error. My matrices contain data.

tan5251 avatar Mar 22 '23 01:03 tan5251

hi, I'm having the same issue with this function. I'm using .cools.

LucasMcNU avatar Mar 03 '24 18:03 LucasMcNU

this is the error I'm receiving:

` INFO:numexpr.utils:Note: NumExpr detected 52 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. INFO:numexpr.utils:NumExpr defaulting to 8 threads. INFO:hicexplorer.hicCorrelate:loading hic matrix /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MB1/mb1.cool

INFO:hicexplorer.hicCorrelate:restore masked bins /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MB1/mb1.cool

INFO:hicexplorer.hicCorrelate:loading hic matrix /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MB2/mb2.cool

INFO:hicexplorer.hicCorrelate:restore masked bins /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MB2/mb2.cool

INFO:hicexplorer.hicCorrelate:loading hic matrix /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MT1/mt1.cool

INFO:hicexplorer.hicCorrelate:restore masked bins /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MT1/mt1.cool

INFO:hicexplorer.hicCorrelate:loading hic matrix /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MT2/mt2.cool

INFO:hicexplorer.hicCorrelate:restore masked bins /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MT2/mt2.cool

Traceback (most recent call last): File "/home/lmc0633/.local/bin/hicCorrelate", line 7, in main() File "/home/lmc0633/.local/lib/python3.6/site-packages/hicexplorer/hicCorrelate.py", line 330, in main rows_keep = cols_keep = np.delete(list(range(all_mat.shape[1])), all_nan) File "<array_function internals>", line 6, in delete File "/home/lmc0633/.local/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4406, in delete keep[obj,] = False IndexError: arrays used as indices must be of integer (or boolean) type `

LucasMcNU avatar Mar 03 '24 22:03 LucasMcNU

Which HiCExplorer version? You run also Python 3.6 which we do not support anymore. Please make sure to run the latest version 3.7.3 and a supported Python version.

LucasMcNU @.***> schrieb am So. 3. März 2024 um 23:03:

this is the error I'm receiving:

`INFO:numexpr.utils:Note: NumExpr detected 52 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. INFO:numexpr.utils:NumExpr defaulting to 8 threads. INFO:hicexplorer.hicCorrelate:loading hic matrix /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MB1/mb1.cool

INFO:hicexplorer.hicCorrelate:restore masked bins /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MB1/mb1.cool

INFO:hicexplorer.hicCorrelate:loading hic matrix /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MB2/mb2.cool

INFO:hicexplorer.hicCorrelate:restore masked bins /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MB2/mb2.cool

INFO:hicexplorer.hicCorrelate:loading hic matrix /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MT1/mt1.cool

INFO:hicexplorer.hicCorrelate:restore masked bins /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MT1/mt1.cool

INFO:hicexplorer.hicCorrelate:loading hic matrix /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MT2/mt2.cool

INFO:hicexplorer.hicCorrelate:restore masked bins /projects/b1103/HIC_Macquarrie/Lucas/cool_reps/MT2/mt2.cool

Traceback (most recent call last): File "/home/lmc0633/.local/bin/hicCorrelate", line 7, in main() File "/home/lmc0633/.local/lib/python3.6/site-packages/hicexplorer/hicCorrelate.py", line 330, in main rows_keep = cols_keep = np.delete(list(range(all_mat.shape[1])), all_nan) File "<array_function internals>", line 6, in delete File "/home/lmc0633/.local/lib/python3.6/site-packages/numpy/lib/function_base.py", line 4406, in delete keep[obj,] = False IndexError: arrays used as indices must be of integer (or boolean) type`

— Reply to this email directly, view it on GitHub https://github.com/deeptools/HiCExplorer/issues/782#issuecomment-1975373491, or unsubscribe https://github.com/notifications/unsubscribe-auth/ADGQCAHXJXENFIZCPI6ZF2TYWOMZVAVCNFSM5PJ4SKT2U5DIOJSWCZC7NNSXTN2JONZXKZKDN5WW2ZLOOQ5TCOJXGUZTOMZUHEYQ . You are receiving this because you commented.Message ID: @.***>

joachimwolff avatar Mar 03 '24 22:03 joachimwolff

I updated to the most recent version and now have the following issue:

Traceback (most recent call last): File "/home/lmc0633/.conda/envs/hicexplorer/bin/hicCorrelate", line 7, in main() File "/home/lmc0633/.conda/envs/hicexplorer/lib/python3.10/site-packages/hicexplorer/hicCorrelate.py", line 394, in main results[row, col] = correlation_opts[args.method](vector1, vector2)[0] File "/home/lmc0633/.local/lib/python3.10/site-packages/scipy/stats/_stats_py.py", line 4727, in pearsonr raise ValueError('x and y must have length at least 2.') ValueError: x and y must have length at least 2.

LucasMcNU avatar Mar 05 '24 04:03 LucasMcNU