WorldBrain
Skip pages with basic http auth during importing
When you encounter sites requiring basic http authentication during the import the whole process stops and waits. So maybe just skip these pages altogether.
Want to back this issue? Post a bounty on it! We accept bounties via Bountysource.
Thanks @dionysio for your request.
Do you have an idea on how to easily identify them?
You could check for 401 Unauthorized status first (and possibly WWW-Authenticate header for basic/digest). Or maybe only accept 20x codes.
I wonder, why they are actually called then, because we already filter out all requests that are !=200
if (xhttp.readyState == 4 && xhttp.status == 200)
if (xhttp.readyState == 4 && xhttp.status == 200)
The line you've posted is exactly what I meant - that it should only process responses that have 200 status... so I'm not sure why they actually pass that.
@dionysio @oliversauter it might be that the code drops that 401 and don't process it but browser just shows login window which block further processing?
