Autometa icon indicating copy to clipboard operation
Autometa copied to clipboard

ncbi rsync error when downloading markers

Open hyphaltip opened this issue 2 years ago • 0 comments

Current Behavior

Error message about ncbi server connection but this is a marker download (raw.github.com download)

  File "/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/lib/python3.10/site-packages/autometa/config/databases.py", line 204, in get_remote_checksum
    raise ConnectionError("Cannot connect to the NCBI rsync server")
ConnectionError: Cannot connect to the NCBI rsync server

Steps to Reproduce

autometa-update-databases --update-markers

Expected Behavior

This is an unnecessary error since opening the rsync connection is not happening for this marker db.

Environment Information

autometa-config --print

[10/31/2023 08:43:50 AM DEBUG] root: environment dependencies satisifed: True
section	option	value
common	home_dir	/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/lib/python3.10/site-packages
environ	diamond	/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/bin/diamond
environ	hmmsearch	/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/bin/hmmsearch
environ	hmmpress	/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/bin/hmmpress
environ	hmmscan	/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/bin/hmmscan
environ	prodigal	/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/bin/prodigal
environ	bowtie2	/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/bin/bowtie2
environ	samtools	/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/bin/samtools
environ	bedtools	/bigdata/operations/pkgadmin/opt/linux/centos/8.x/x86_64/pkgs/autometa/2.2.1/env/bin/bedtools
versions	diamond	2.1.8
versions	hmmsearch	3.3.2
versions	hmmpress	3.3.2
versions	hmmscan	3.3.2
versions	prodigal	2.6.3
versions	bowtie2	2.5.2
versions	samtools	1.18
versions	bedtools	2.31.0
databases	base	/srv/projects/db/autometa/2.0/
databases	ncbi	/srv/projects/db/autometa/2.0
databases	gtdb	/srv/projects/db/autometa/2.0/gtdb/
databases	markers	/srv/projects/db/autometa/2.0/markers/
database_urls	taxdump	ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz
database_urls	accession2taxid	ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/accession2taxid/prot.accession2taxid.gz
database_urls	nr	ftp://ftp.ncbi.nlm.nih.gov/blast/db/FASTA/nr.gz
database_urls	bacteria_single_copy	https://raw.githubusercontent.com/KwanLab/Autometa/main/autometa/databases/markers/bacteria.single_copy.hmm
database_urls	bacteria_single_copy_cutoffs	https://raw.githubusercontent.com/KwanLab/Autometa/main/autometa/databases/markers/bacteria.single_copy.cutoffs
database_urls	archaea_single_copy	https://raw.githubusercontent.com/KwanLab/Autometa/main/autometa/databases/markers/archaea.single_copy.hmm
database_urls	archaea_single_copy_cutoffs	https://raw.githubusercontent.com/KwanLab/Autometa/main/autometa/databases/markers/archaea.single_copy.cutoffs
database_urls	proteins_aa_reps	https://data.gtdb.ecogenomic.org/releases/latest/genomic_files_reps/gtdb_proteins_aa_reps.tar.gz
database_urls	gtdb_taxdmp	https://github.com/shenwei356/gtdb-taxdump/releases/latest/download/gtdb-taxdump.tar.gz
checksums	taxdump	ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/taxdump.tar.gz.md5
checksums	accession2taxid	ftp://ftp.ncbi.nlm.nih.gov/pub/taxonomy/accession2taxid/prot.accession2taxid.gz.md5
checksums	nr	ftp://ftp.ncbi.nlm.nih.gov/blast/db/FASTA/nr.gz.md5
checksums	bacteria_single_copy	https://raw.githubusercontent.com/KwanLab/Autometa/main/autometa/databases/markers/bacteria.single_copy.hmm.md5
checksums	bacteria_single_copy_cutoffs	https://raw.githubusercontent.com/KwanLab/Autometa/main/autometa/databases/markers/bacteria.single_copy.cutoffs.md5
checksums	archaea_single_copy	https://raw.githubusercontent.com/KwanLab/Autometa/main/autometa/databases/markers/archaea.single_copy.hmm.md5
checksums	archaea_single_copy_cutoffs	https://raw.githubusercontent.com/KwanLab/Autometa/main/autometa/databases/markers/archaea.single_copy.cutoffs.md5
ncbi	host	ftp.ncbi.nlm.nih.gov
ncbi	taxdump	/srv/projects/db/autometa/2.0/taxdump.tar.gz
ncbi	nodes	/srv/projects/db/autometa/2.0/nodes.dmp
ncbi	names	/srv/projects/db/autometa/2.0/names.dmp
ncbi	merged	/srv/projects/db/autometa/2.0/merged.dmp
ncbi	delnodes	/srv/projects/db/autometa/2.0/delnodes.dmp
ncbi	accession2taxid	/srv/projects/db/autometa/2.0/prot.accession2taxid.gz
ncbi	nr	/srv/projects/db/autometa/2.0/nr.gz
gtdb	host	data.gtdb.ecogenomic.org
gtdb	release	latest
gtdb	proteins_aa_reps	/srv/projects/db/autometa/2.0/gtdb//gtdb_proteins_aa_reps.tar.gz
gtdb	gtdb_taxdmp	/srv/projects/db/autometa/2.0/gtdb//gtdb-taxdump.tar.gz
markers	host	raw.githubusercontent.com
markers	bacteria_single_copy	/srv/projects/db/autometa/2.0/markers//bacteria.single_copy.hmm
markers	bacteria_single_copy_cutoffs	/srv/projects/db/autometa/2.0/markers//bacteria.single_copy.cutoffs
markers	archaea_single_copy	/srv/projects/db/autometa/2.0/markers//archaea.single_copy.hmm
markers	archaea_single_copy_cutoffs	/srv/projects/db/autometa/2.0/markers//archaea.single_copy.cutoffs
files	metagenome	metagenome.fna
files	fwd_reads	fwd_reads.fastq
files	rev_reads	rev_reads.fastq
files	se_reads	se_reads.fastq
files	sam	alignments.sam
files	bam	alignments.bam
files	lengths	lengths.tsv
files	bed	alignments.bed
files	length_filtered	metagenome.filtered.fna
files	coverages	coverages.tsv
files	kmer_counts	kmers.tsv
files	kmer_normalized	kmers.normalized.tsv
files	kmer_embedded	kmers.embedded.tsv
files	nucleotide_orfs	metagenome.filtered.orfs.fna
files	amino_acid_orfs	metagenome.filtered.orfs.faa
files	blastp	blastp.tsv
files	blastp_hits	blastp.hits.pkl.gz
files	lca	lca.tsv
files	blastx	blastx.tsv
files	taxonomy	taxonomy.tsv
files	bacteria_hmmscan	bacteria.hmmscan.tsv
files	bacteria_markers	bacteria.markers.tsv
files	archaea_hmmscan	archaea.hmmscan.tsv
files	archaea_markers	archaea.markers.tsv
files	bacteria_binning	bacteria.binning.tsv
files	archaea_binning	archaea.binning.tsv
files	checkpoints	checkpoints.tsv

hyphaltip avatar Oct 31 '23 15:10 hyphaltip