PyPaperBot icon indicating copy to clipboard operation
PyPaperBot copied to clipboard

Problem to fetch 100 PDFS using this package

Open sreejata-b opened this issue 4 years ago • 1 comments

Hello.

I am trying to download 100 pdfs using dois using pyPaperBot. But only 41 gets downloaded and i get this error.

Here are the error messages. It finished download at number 40 and then printed this : TypeError: expected string or bytes-like object. detailed error below. Thanks a lot in advance.


Download 40 of 100 -> Biochar decreased rhizodeposits stabilization via opposite effects on bacteria and fungi: diminished fungi-promoted aggregation and enhanced bacterial mineralization Download 41 of 100 -> None Traceback (most recent call last): File "/mnt/home/bandopad/miniconda3/lib/python3.7/runpy.py", line 193, in run_module_as_main "main", mod_spec) File "/mnt/home/bandopad/miniconda3/lib/python3.7/runpy.py", line 85, in run_code exec(code, run_globals) File "/mnt/ufs18/rs-033/ShadeLab/WorkingSpace/Bandopadhyay_WorkingSpace/metaanalysis_doi/environment/lib/python3.7/site-packages/PyPaperBot/main.py", line 122, in main() File "/mnt/ufs18/rs-033/ShadeLab/WorkingSpace/Bandopadhyay_WorkingSpace/metaanalysis_doi/environment/lib/python3.7/site-packages/PyPaperBot/main.py", line 118, in main start(args.query, args.scholar_pages, dwn_dir, args.min_year , max_dwn, max_dwn_type , args.journal_filter, args.restrict, DOIs) File "/mnt/ufs18/rs-033/ShadeLab/WorkingSpace/Bandopadhyay_WorkingSpace/metaanalysis_doi/environment/lib/python3.7/site-packages/PyPaperBot/main.py", line 45, in start downloadPapers(to_download, dwn_dir, num_limit) File "/mnt/ufs18/rs-033/ShadeLab/WorkingSpace/Bandopadhyay_WorkingSpace/metaanalysis_doi/environment/lib/python3.7/site-packages/PyPaperBot/Downloader.py", line 62, in downloadPapers pdf_dir = getSaveDir(dwnl_dir, p.getFileName()) File "/mnt/ufs18/rs-033/ShadeLab/WorkingSpace/Bandopadhyay_WorkingSpace/metaanalysis_doi/environment/lib/python3.7/site-packages/PyPaperBot/Paper.py", line 31, in getFileName return re.sub('[^\w-. ]', '', self.title)+".pdf" File "/mnt/home/bandopad/miniconda3/lib/python3.7/re.py", line 192, in sub return _compile(pattern, flags).sub(repl, string, count) TypeError: expected string or bytes-like object (environment) (base) -bash-4.2$

sreejata-b avatar Jun 03 '21 14:06 sreejata-b

Hi, sorry for the late reply, but PyPaperBot had been on standby until now. I've decided to resume development to improve it.

I've created a Telegram channel where you can suggest improvements, report bugs, or request custom data mining scripts. Feel free to join if you're interested with the following LINK

Thanks!

ferru97 avatar Oct 07 '24 22:10 ferru97