pingora icon indicating copy to clipboard operation
pingora copied to clipboard

Disable Chunk Encoding by Config

Open janbraunsdorff opened this issue 1 year ago • 4 comments

What is the problem your feature solves, or the need it fulfills?

Bevor Upstream the content gets parse base on the corrosponding headers. Some Clients still use http1 with Chunk encoding. Is there a way to disabel Chunk encoding parsing for all requests from Downstream?

Describe the solution you'd like

A server Config to default disabele Chunk Encoding in the yaml file.

Describe alternatives you've considered

Solve the issue client Side or use an nginx befor the proxy.

Additional context

AWS S3 clients and pyspark still uses Chunk Enconding on http1. During access throw the proxy, the request will fail because the md5-hash is incorrect. Therefore is there way to pass the payload throw without converting it?

Or am I completly wrong and miss something?

janbraunsdorff avatar Apr 12 '24 19:04 janbraunsdorff

Do you mean that you want a feature to preserve the exact request chunked encoding through our proxy? Or do you what a feature to convert chunked encoding to something else (content-length)?

eaufavor avatar Apr 18 '24 22:04 eaufavor

I want to have a feature to pass throw the chunked encoding.

client --> Proxy --> Server. The Server needs the exact encoding from the client. No buffering or Parsing. So the proxy only reads the header, do something (e.g. Routing and Auth) an pass throw the body without parsing oder try to read all chunks

janbraunsdorff avatar Apr 22 '24 19:04 janbraunsdorff

Hi everyone

for more infromation, i try to build an Gateway for Authentification for an S3 Storage. Older Clients want to use chunked Encoding. And the Chunks will be reassembled by S3. So I need an option to pass throw the raw payload. Is this possible?

Thank you :)

janbraunsdorff avatar Apr 29 '24 08:04 janbraunsdorff

It is possible for us to implement something that doesn't combine chunks internally in order to preserve. Because this is a relatively niche ask, it has not been worked on yet.

eaufavor avatar May 01 '24 18:05 eaufavor