proxy mode for http proxies

[T-256]

Closes #735 Refs: https://github.com/encode/httpcore/discussions/703 https://github.com/encode/httpcore/pull/716

• [x] tests (forward and tunnel tests already implemented)
• [x] docs + warn about insecure use of ProxyMode.FORWARD
• [x] changelog

[T-256]

current implementation is modified and upgraded of first option discussed in https://github.com/encode/httpcore/discussions/703#discussioncomment-6201442. for second option we can use only one flagged param:

class ProxyMode(enum.IntFlag):
    HTTPS_FORWARD = 1
    HTTP_TUNNEL = 2


def HTTPProxy(
    ...
    mode: Optional[ProxyMode] = None,
): ...

EDIT: Now I think, this would be better option (using one param instead of two), going for replace it.

[karpetrosyan]

We should probably wait for #714, then open the issue, and only then go through this for further clarity.

[T-256]

We should probably wait for #714, then open the issue, and only then go through this for further clarity.

I didn't understand. Will #714 create any blocker for this PR or vice versa? Actually it just adds TLS for connecting to proxy server, both forwarding and tunneling will sent normally after TLS stablished. So, IMO it is just extra layer and should be compatible with this PR.

[karpetrosyan]

I've started an issue for this PR.

[karpetrosyan]

Let's change the title of this PR to something like "Add force-tunneling and force-forwarding support for proxies."

[lovelydinosaur]

Really I'd prefer that we work through design discussions first before starting on the pull request, but...

• I'm okay with reviewing this once there's docs.
• We also need to consider what the implications/expectations are for pushing this downstream to httpx.

[T-256]

Any new thoughts here? it stuck on implementation design. few I can suggest are:

• separated parameters for http and https (implemented in here)
• one flagged parameter (current implementation)
• one routing parameter: dict[str, Literal["TUNNEL"] | Literal["FORWARD"]] (like in httpx)

And if you have new idea these are things should be considered to work:

• don't change current default: forward http and tunnel https
• able to set force forward or force tunnel
• opposite of current default: tunnel http and forward https (maybe not mandatory? any use cases?)

[T-256]

async cancellation test failing randomly. not related to this PR.

[T-256]

According to Mozzilla:

There are two types of proxies: forward proxies (or tunnel, or gateway) and reverse proxies (used to control and protect access to a server for load-balancing, authentication, decryption or caching).

So, should we change the ProxyMode.FORWARD name? Which one is proper name?

• ABSOLUTE_URL
• ABSOLUTE
• PLAIN
• GATEWAY

[T-256]

Is it possible to include in https://github.com/encode/httpcore/pull/768?

[lovelydinosaur]

So...

1. FORWARD/TUNNEL makes more sense to me. That's a reading that'd make sense to me from these docs. It's also consistent with urllib3, and our httpx docs.
2. I'm not fully convinced yet that we couldn't instead have a use_forwarding_for_https flag. That's the API that urllib3 use for this manual control, and they don't seem to have open issues suggesting that "allow CONNECT tunnelling to http" is something their users are missing. I can see that we discussed this briefly at https://github.com/encode/httpcore/discussions/703#discussioncomment-6174581. Is someone able to provide an example of a proxy that would require this configuration, so I'm able to test it out and verify that we really do want to support that as a configuration option?

[karpetrosyan]

1. Completely agree
2. If the proxy server supports tunneling and it's not a public proxy, then using tunneling over HTTP makes sense. Assume that we are using a local HTTP proxy server that supports tunneling. We do not need to configure certificates and make the proxy server secure because we actually trust it.

Example

  trusted
host -> proxy

   not trusted
proxy -> remote server

| Solution |

HTTP tunneling gives
      http       https
host   ->   proxy -> remote-server

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.