pylinkvalidator icon indicating copy to clipboard operation
pylinkvalidator copied to clipboard

Fix issue with Base64 encoding for basic auth

Open airsource opened this issue 6 years ago • 0 comments

Simple fix to resolve the newline encoded at the end of the basic auth string.

Symptom:

ERROR Crawled 1 urls with 1 error(s) in 0.01 seconds

  Start URL(s): http://foobar.foo

  error (<type 'exceptions.ValueError'>): Invalid header value 'Basic YWlyc291cmNlOmJsYWhibGFo\n': http://foobar.foo

Cause:

As others have reported (e.g. here there's a problem with using the encode methods for Base64 encoding as they append a newline character, which results in an invalid header. This allegedly started to occur after a security fix made to Python.

Fix:

Replace the Base64 encode method with b64encode instead - this doesn't append a newline.

airsource avatar May 17 '19 11:05 airsource