Gerd

I can't find anything in the documentation about how to use mongodb to save the crawled data. Am I missing something ?

scraping https endpoint with invalid cert

1

How can I add tls.verify to false?

annoying warning "Syntax Warning: Could not parse ligature component" and possible solution to suppress these messages..

1

I am getting such warnings on some pdf-s that come from pdftotext and originate from poppler, these messages are in stderr and pdftotext has an option to suppress them with...