w3lib

w3lib copied to clipboard

Hi, I hope I find you well. I found the following issue, not sure if it's a bug, but it didn't behave like this in the previous version (1.22). Before...

azierariel

safe_url_string URL-encodes already-encoded username and password, breaking idempodency

1

The documentation claims that calling `safe_url_string` on an already “safe” URL will return the URL unmodified, but this breaks when the username or password include `%`. ```python >>> url =...

andersk

It's been over 2 years. Please create a new tagged release for [PyPi](https://pypi.org/project/w3lib/).

yozachar

I have reason to believe that `basic_auth_header` is wrong in using `urlsafe_b64encode` (which replaces `+/` with `-_`) instead of `b64encode`. The first specification of HTTP basic auth according to [Wikipedia](https://en.wikipedia.org/wiki/Basic_access_authentication)...

Gallaecio

Strip spaces in canonicalize_url

3

Fixes #132

Gallaecio

Fix a bug for header field getting malformed when it extends over multiple lines

6

The message headers section of RFC 2616 (Hypertext Transfer Protocol -- HTTP/1.1) states that > Header fields can be extended over multiple lines by preceding each extra line with at...

okomestudio

[MRG+1] Import scurl if found installed

9

Hey @lopuhin, I made this PR to run scurl test on w3lib itself. Let me know if there's something that can be improved here 😄

malloxpb

fix(url): canonicalize_url('http://您好.中国:80/') failed

6

expect: `'http://xn--5usr0o.xn--fiqs8s:80/'` actual: `'http://xn--5usr0o.xn--:80-u68dy61b/'`

hidva

[MRG+1] Added: Removing comments before extracting base URLs. Not a solution to #70, but does help in some cases.

6

Helps resolving the issue in such cases, which does happen in several websites: ``` Python >>> from w3lib import html >>> html.get_base_url("""""") 'http://example.com/' ``` Fixes #70 (since the original #70...

starrify

Expose the separator parameter added upstream to parse_qs and parse_qsl

3

Gallaecio