cli
cli copied to clipboard
httpie
httpie -d does not work with gzip compressed content
Checklist
- [x] I've searched for similar issues.
- [x] I'm using the latest version of HTTPie.
Minimal reproduction code and steps
Expose an API endpoint that respond with gzip compressed json such as an Azure storage account.
- http get -d -o foo.json https://.../path/to/endpoint
Current result
+ http -d -I --check-status GET https://XXXX.blob.core.windows.net/invoices/p32132/1201394/input.json?sv=2021-12-02&se=2024-02-25T19%3A08%3A46Z&sr=c&sp=r&sig=XXXXX -o p32132/1201394/input.json
HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: x-ms-request-id,Server,x-ms-version,x-ms-resource-type,Content-Type,Content-Encoding,Last-Modified,ETag,x-ms-creation-time,Content-MD5,x-ms-lease-status,x-ms-lease-state,x-ms-blob-type,x-ms-server-encrypted,Accept-Ranges,x-ms-owner,x-ms-group,x-ms-permissions,Content-Length,Date,Transfer-Encoding
Content-Encoding: gzip
Content-Length: 55119
Content-MD5: IJ8A578M7DLY5/jW+7umKw==
Content-Type: application/json
Date: Fri, 26 Jan 2024 19:08:46 GMT
ETag: "0x8DBFA81678EA69D"
Last-Modified: Mon, 11 Dec 2023 19:43:09 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-blob-type: BlockBlob
x-ms-creation-time: Mon, 11 Dec 2023 19:36:34 GMT
x-ms-group: $superuser
x-ms-lease-state: available
x-ms-lease-status: unlocked
x-ms-owner: $superuser
x-ms-permissions: rw-r-----
x-ms-request-id: c3159c91-001e-0064-578b-5091a4000000
x-ms-resource-type: file
x-ms-server-encrypted: true
x-ms-version: 2021-12-02
Downloading to p32132/1201394/input.json
Done. 118.9 kB in 00:0.08901 (1.3 MB/s)
http: LogLevel.ERROR: Incomplete download: size=55119; downloaded=118911
…
Expected result
Httpie should consider the compressed size to validate that the entire content was downloaded, not consider the uncompressed size with the compressed gzip payload that was sent.
…
Debug output
Please re-run the command with --debug, then copy the entire command & output and paste both below:
+ http -d --debug -I --check-status GET https://XXXX.blob.core.windows.net/invoices/p32132/1201394/input.json?sv=2021-12-02&se=2024-02-25T19%3A10%3A41Z&sr=c&sp=r&sig=XXXX -o p32132/1201394/input.json
HTTPie 3.2.1
Requests 2.31.0
Pygments 2.17.2
Python 3.11.6 (main, Nov 14 2023, 18:04:26) [GCC 13.2.1 20230801]
/usr/bin/python3
Linux 6.7.0-zen3-1-zen
<Environment {'apply_warnings_filter': <function Environment.apply_warnings_filter at 0x78780a9fd120>,
'args': Namespace(),
'as_silent': <function Environment.as_silent at 0x78780a9fcfe0>,
'colors': 256,
'config': {'default_options': []},
'config_dir': PosixPath('/home/root/.config/httpie'),
'devnull': <property object at 0x78780a9ed4e0>,
'is_windows': False,
'log_error': <function Environment.log_error at 0x78780a9fd080>,
'program_name': 'http',
'quiet': 0,
'rich_console': <functools.cached_property object at 0x78780a9f3050>,
'rich_error_console': <functools.cached_property object at 0x78780a9f3150>,
'show_displays': True,
'stderr': <_io.TextIOWrapper name='<stderr>' mode='w' encoding='utf-8'>,
'stderr_isatty': True,
'stdin': <_io.TextIOWrapper name='<stdin>' mode='r' encoding='utf-8'>,
'stdin_encoding': 'utf-8',
'stdin_isatty': False,
'stdout': <_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>,
'stdout_encoding': 'utf-8',
'stdout_isatty': True}>
<PluginManager {'adapters': [],
'auth': [<class 'httpie.plugins.builtin.BasicAuthPlugin'>,
<class 'httpie.plugins.builtin.DigestAuthPlugin'>,
<class 'httpie.plugins.builtin.BearerAuthPlugin'>],
'converters': [],
'formatters': [<class 'httpie.output.formatters.headers.HeadersFormatter'>,
<class 'httpie.output.formatters.json.JSONFormatter'>,
<class 'httpie.output.formatters.xml.XMLFormatter'>,
<class 'httpie.output.formatters.colors.ColorFormatter'>]}>
>>> requests.request(**{'auth': None,
'data': RequestJSONDataDict(),
'headers': <HTTPHeadersDict('User-Agent': b'HTTPie/3.2.1', 'Accept-Encoding': b'identity')>,
'method': 'get',
'params': <generator object MultiValueOrderedDict.items at 0x78780bca6a40>,
'url': 'https://XXXX.blob.core.windows.net/invoices/p32132/1201394/input.json?sv=2021-12-02&se=2024-02-25T19%3A10%3A41Z&sr=c&sp=r&sig=XXXX'})
HTTP/1.1 200 OK
Accept-Ranges: bytes
Access-Control-Allow-Origin: *
Access-Control-Expose-Headers: x-ms-request-id,Server,x-ms-version,x-ms-resource-type,Content-Type,Content-Encoding,Last-Modified,ETag,x-ms-creation-time,Content-MD5,x-ms-lease-status,x-ms-lease-state,x-ms-blob-type,x-ms-server-encrypted,Accept-Ranges,x-ms-owner,x-ms-group,x-ms-permissions,Content-Length,Date,Transfer-Encoding
Content-Encoding: gzip
Content-Length: 55119
Content-MD5: IJ8A578M7DLY5/jW+7umKw==
Content-Type: application/json
Date: Fri, 26 Jan 2024 19:10:41 GMT
ETag: "0x8DBFA81678EA69D"
Last-Modified: Mon, 11 Dec 2023 19:43:09 GMT
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-blob-type: BlockBlob
x-ms-creation-time: Mon, 11 Dec 2023 19:36:34 GMT
x-ms-group: $superuser
x-ms-lease-state: available
x-ms-lease-status: unlocked
x-ms-owner: $superuser
x-ms-permissions: rw-r-----
x-ms-request-id: c8568e21-501e-0079-548b-509c18000000
x-ms-resource-type: file
x-ms-server-encrypted: true
x-ms-version: 2021-12-02
Downloading to p32132/1201394/input.json
Done. 118.9 kB in 00:0.08444 (1.4 MB/s)
http: LogLevel.ERROR: Incomplete download: size=55119; downloaded=118911
Additional information, screenshots, or code examples
…
