ChaissonLab
Error in Build RPGG for my own datasets
Hi, I use 35 hyplotypes to build RPGG for my VNTR sets, I find that the program always reports the following error when I run /soft/danbing-tk-1.1//script/preMBE.py /35Hyplto/genome.bam.tsv pan.tr.mbe.v0.bed: beds[hi,m] = np.loadtxt(f"{g}/tmp1.{h}.bed", dtype=object, usecols=[0,1,2,6], ndmin=2) ValueError: shape mismatch: value array of shape (101497,4) could not be broadcast to indexing result of shape (37920,4) I checked f"{g}/tmp1.{h}.bed" file, it's seems that every hyplotype file have different number of mapping locations, what should I do to solve this problem, any suggestion will be appreciated.
Best wishes Lee
Hi Lee,
Could you trying using the latest commit? The preMBE.py is obsolete and not used in the current pipeline.
Cheers, -Tony
Hi Tony, I tried both danbing-tk-1.3.2 and danbing-tk-1.0, I find that danbing-tk-1.0 can run successfully 'snakemake -p -s ../pipeline/GoodPanGenomeGraph.snakefile -j 4 --rerun-incomplete --output-wait 3', but danbing-tk-1.3.2 always have the following message: module load gcc
/rd1/laixh/soft/danbing-tk-1.3.2//bin/vntr2kmers_thread -g -m <(cut -f $((0+1)),$((0+2)) /rd1/laixh/soft/danbing-tk-1.3. 2/test/output/OrthoMap.v2.tsv) -k 21 -fs 700 -ntr 700 -on HG00514.rawPB -fa 2 /rd1/laixh/soft/danbing-tk-1.3.2/test/outp ut/HG00514.0.tr.fasta /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00514.1.tr.fasta
if [ 0 == "1" ]; then samtools fasta -@2 -n | /rd1/laixh/soft/danbing-tk-1.3.2//bin/bam2pe -fai /dev/stdin | /rd1/laixh/soft/danbing-tk-1.3.2//bin/danbing-tk -g 50 -k 21 -qs /rd1/laixh/soft/danbing-tk-1.3.2/test/output//HG005 14.rawPB -fai /dev/stdin -o HG00514.rawIL -p 1 -cth 45 -rth 0.5 fi
/bin/bash: line 3: module: command not found [Thu Jul 20 11:54:39 2023] Error in rule GenRawGenomeGraph: jobid: 11 output: /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.tr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/o utput/HG00733.rawPB.ntr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.graph.kmers shell:
set -eu ulimit -c 20000 cd /rd1/laixh/soft/danbing-tk-1.3.2/test/output/ module load gcc
/rd1/laixh/soft/danbing-tk-1.3.2//bin/vntr2kmers_thread -g -m <(cut -f $((2+1)),$((2+2)) /rd1/laixh/soft/danbing-tk-1.3. 2/test/output/OrthoMap.v2.tsv) -k 21 -fs 700 -ntr 700 -on HG00733.rawPB -fa 2 /rd1/laixh/soft/danbing-tk-1.3.2/test/outp ut/HG00733.0.tr.fasta /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.1.tr.fasta
if [ 0 == "1" ]; then samtools fasta -@2 -n | /rd1/laixh/soft/danbing-tk-1.3.2//bin/bam2pe -fai /dev/stdin | /rd1/laixh/soft/danbing-tk-1.3.2//bin/danbing-tk -g 50 -k 21 -qs /rd1/laixh/soft/danbing-tk-1.3.2/test/output//HG007 33.rawPB -fai /dev/stdin -o HG00733.rawIL -p 1 -cth 45 -rth 0.5 fi
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
/bin/bash: line 3: module: command not found [Thu Jul 20 11:54:39 2023] Error in rule GenRawGenomeGraph: jobid: 10 output: /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00514.rawPB.tr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/o utput/HG00514.rawPB.ntr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00514.rawPB.graph.kmers shell:
set -eu ulimit -c 20000 cd /rd1/laixh/soft/danbing-tk-1.3.2/test/output/ module load gcc
/rd1/laixh/soft/danbing-tk-1.3.2//bin/vntr2kmers_thread -g -m <(cut -f $((0+1)),$((0+2)) /rd1/laixh/soft/danbing-tk-1.3. 2/test/output/OrthoMap.v2.tsv) -k 21 -fs 700 -ntr 700 -on HG00514.rawPB -fa 2 /rd1/laixh/soft/danbing-tk-1.3.2/test/outp ut/HG00514.0.tr.fasta /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00514.1.tr.fasta
if [ 0 == "1" ]; then samtools fasta -@2 -n | /rd1/laixh/soft/danbing-tk-1.3.2//bin/bam2pe -fai /dev/stdin | /rd1/laixh/soft/danbing-tk-1.3.2//bin/danbing-tk -g 50 -k 21 -qs /rd1/laixh/soft/danbing-tk-1.3.2/test/output//HG005 14.rawPB -fai /dev/stdin -o HG00514.rawIL -p 1 -cth 45 -rth 0.5 fi
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Shutting down, this might take some time. Exiting because a job execution failed. Look above for error message
It's seems that lack of module make the command failed. Because our platform do not support this software, so I delete all the module load command in file GoodPanGenomeGraph.snakefile.
The pan.tr.kmers, pan.ntr.kmers and pan.graph.kmers successfully generated but error message occur when run rule GenSerializedGraphAndIndex, error message below:
Error in rule GenSerializedGraphAndIndex: jobid: 9 output: /rd1/laixh/soft/danbing-tk-1.3.2/test/output/pan.graph.umap, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/pan.kmerDBi.umap, /rd1/laixh/soft/danbi shell:
cd /rd1/laixh/soft/danbing-tk-1.3.2/test/output/ ulimit -c 20000
/rd1/laixh/soft/danbing-tk-1.3.2//bin/ktools serialize /rd1/laixh/soft/danbing-tk-1.3.2/test/output//pan /rd1/laixh/soft/danbing-tk-1.3.2//bin/ktools ksi pan.tr.kmers >/rd1/laixh/soft/danbing-tk-1.3.2/test/output//pan.tr.ksi
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
**I tried command line "/rd1/laixh/soft/danbing-tk-1.3.2//bin/ktools serialize /rd1/laixh/soft/danbing-tk-1.3.2/test/output//pan", this command returns Segmentation fault (core dumped). Do you have any idea to fix this problem?
best wishes, Lee
Hi Lee,
Thanks for reporting this error. I just removed the strict dependency for module. It will now invoke module only when your system supports it. Let me know if this works.
Thanks! -Tony
Hi, Thanks for your reply, but the error message still occur: Error in rule GenRawGenomeGraph: jobid: 11 output: /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.tr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.ntr.kmers, /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.rawPB.graph.kmers shell:
set -eu ulimit -c 20000 cd /rd1/laixh/soft/danbing-tk-1.3.2/test/output/ type module &>/dev/null if [ $? == 0 ]; then module load gcc fi
/rd1/laixh/soft/danbing-tk-1.3.2//bin/vntr2kmers_thread -g -m <(cut -f $((2+1)),$((2+2)) /rd1/laixh/soft/danbing-tk-1.3.2/test/output/OrthoMap.v2.tsv) -k 21 -fs 700 -ntr 700 -on HG00733.rawPB -fa 2 /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.0.tr.fasta /rd1/laixh/soft/danbing-tk-1.3.2/test/output/HG00733.1.tr.fasta
if [ 0 == "1" ]; then samtools fasta -@2 -n | /rd1/laixh/soft/danbing-tk-1.3.2//bin/bam2pe -fai /dev/stdin | /rd1/laixh/soft/danbing-tk-1.3.2//bin/danbing-tk -g 50 -k 21 -qs /rd1/laixh/soft/danbing-tk-1.3.2/test/output//HG00733.rawPB -fai /dev/stdin -o HG00733.rawIL -p 1 -cth 45 -rth 0.5 fi
(one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
Best wishes, Lee
Fixed again! Forgot to make sure return code is zero. Hope it works now.