ArchiveTeam
"%7E" in Location: header is unescaped to raw tilde instead of kept as-is
When wget-lua follows a redirect with a "%7E" in the Location header, it unescapes the %7E into a tilde before sending it. This can be demonstrated as follows (somewhat convoluted, but it's clear what's happening):
- Set up one terminal with
nc -lp 5555 - Set up another with
nc -lp 5557 - In a third, do
wget http://127.0.0.1:5555/page.html - Paste the following into the first one (if you're on a Unix system, this isn't proper HTTP because of the newlines, but wget doesn't care):
HTTP/1.1 301 Redirect
Location: http://127.0.0.1:5557/what%2Ewhat.html
Content-Length: 3
eee
- Wget should follow the redirect, and on your other nc instance you will see something like
GET /what%7Ewhat.html HTTP/1.1
User-Agent: Wget/1.21
Accept: */*
Accept-Encoding: identity
Host: 127.0.0.1:5557
Connection: Keep-Alive
on vanilla wget (1.21), but
GET /what~what.html HTTP/1.1
User-Agent: Wget/1.20.3-at.20210504.01 (linux-gnu)
Accept: */*
Accept-Encoding: identity
Host: 127.0.0.1:5557
Connection: Keep-Alive
on wget-lua/wget-at (1.20.3-at.20210504.01).
This caused problems on Bintray, where it unescaped this in the AWS CDN URL that https://dl.bintray.com/lamyj/generic/apt/pool/main/o/odil/python3-odil_0.12.1-1~stretch_amd64.deb (will hopefully be in the Wayback Machine soon) redirected to, which caused the download to fail. Rewby discovered that --no-iri makes wget-at behave properly (i.e. like upstream wget), and this was indeed adopted on the Bintray project, but this fix can't be use everywhere.
The bug report notes
HTTP/1.1 301 Redirect
Location: http://127.0.0.1:5557/what%2Ewhat.html
Content-Length: 3
eee
should be run, however later examples seem to have used %7E instead of %2E.
Assuming %7E is what is meant, this issue does not happen anymore in version 1.21.3-at.20230621.01.
