S3ClientBuilder will not use the system configured https.proxyHost

[kycrow32]

Describe the bug

By default, S3ClientBuilder will not use the system configured https.proxyHost. Instead it uses the system configured http.proxyHost

Expected Behavior

S3ClientBuilder should connect through the proxy specified by https.proxyHost.

Current Behavior

S3ClientBuilder ignores the value in https.proxyHost. If http.proxyHost is configured, it will connect through that proxy even though it is using an https connection.

Reproduction Steps

1. Unzip s3client.zip
2. Build s3client.jar with "gradlew clean build"
3. set up environment variables: AWS_KEY_ID=some aws access key AWS_KEY_SECRET=some aws secret key AWS_REGION=some aws region
4. set up https proxies export _JAVA_OPTIONS="-Dhttps.proxyHost=SOME_PROXY_HOST -Dhttps.proxyPort=SOME_PROXY_PORT"
5. run java -DLOGGING_LEVEL_ORG_APACHE_HC=DEBUG -DSOFTWARE_AMAZON_AWSSDK=DEBUG -jar s3client.jar
6. if the proxy really is necessary, see a failure! Exception in thread "main" software.amazon.awssdk.core.exception.SdkClientException: Unable to execute HTTP request: Connection reset at software.amazon.awssdk.core.exception.SdkClientException$BuilderImpl.build(SdkClientException.java:111) at software.am s3client.zip azon.awssdk.core.exception.SdkClientException.create(SdkClientException.java:47)
7. in the output logs, see that NO proxy was picked up. You will NOT see "ApacheHttpClient - Configuring Proxy. Proxy Host: SOME_PROXY_HOST"
8. set up http proxies export _JAVA_OPTIONS="-Dhttps.proxyHost=SOME_PROXY_HOST -Dhttps.proxyPort=SOME_PROXY_PORT -Dhttp.proxyHost=SOME_PROXY_HOST -Dhttp.proxyPort=SOME_PROXY_PORT"
9. run java -DLOGGING_LEVEL_ORG_APACHE_HC=DEBUG -DSOFTWARE_AMAZON_AWSSDK=DEBUG -jar s3client.jar
10. see a successful list of your s3 buckets!
11. in the output logs, see that a proxy was picked up. You will see "ApacheHttpClient - Configuring Proxy. Proxy Host: SOME_PROXY_HOST"

Possible Solution

No response

Additional Information/Context

It looks like this happens becuase software.amazon.awssdk.http.apache.ProxyConfiguration won't set a host if no endpoint was provided, and an endpoint is not provided by default. So by default, the S3Client (and I assume other aws clients) will connect using https (good), but use the http.proxyHost system setting, instead of the https.proxyHost system setting (bad).

I tried calling s3ClientBuilder.endpointOverride("https://HOST:PORT"), but S3Client continued to fail to connect. And endpointOverride looks like it is expecting an S3 URL anyway.

AWS Java SDK version used

2.20.30

JDK version used

OpenJDK 17.0.6

Operating System and version

Oracle Linux 8.7

[kycrow32]

This seems to be a different issue than those previously reported, but this may be related to: https://github.com/aws/aws-sdk-java/issues/524 https://github.com/aws/aws-sdk-java-v2/issues/751

[debora-ito]

Moving this issue to the Java SDK 2.x repo.

[debora-ito]

@kycrow32 I can repro the issue.

Looking at the code, the https.* system properties will be picked up only if the scheme of the Proxy Configuration endpoint is HTTPS - this will work:

    System.setProperty("https.proxyHost", "localhost");
    System.setProperty("https.proxyPort", Integer.toString(1234));

    ProxyConfiguration proxyConfig = ProxyConfiguration.builder()
        .endpoint(URI.create("https://localhost:1234"))
        .useSystemPropertyValues(true)
        .build();

    S3Client client = S3Client.builder()
        .httpClientBuilder(ApacheHttpClient.builder().proxyConfiguration(proxyConfig))
        .build();

We even have a test for this behavior. I don't know the reasoning behind this logic, but I'll ask the team.

[debora-ito]

Discussed this with the rest of team, and we are treating this as a bug.

Ideally, https.* system properties should have the same behavior as http.*. We'll need to figure out how to fix this in a backwards compatible way.

[MIJOTHY]

We even have a test for this behavior. I don't know the reasoning behind this logic, but I'll ask the team.

Chiming in here as I've run in to the same issue. I'm a bit confused by these tests.

For example, the following is meant to check that system properties aren't being used. But the only property which is present and differs between the inlined .endpoint configuration and the system property is the port (1234 vs 5555 respectively). But there is no assertion against the explicit endpoint configuration overriding the system property. https://github.com/aws/aws-sdk-java-v2/blob/fd0a4c2ae7a7ff234cf61d0d63861094c5cbfa11/http-clients/apache-client/src/test/java/software/amazon/awssdk/http/apache/ProxyConfigurationTest.java#L125-L143

The same applies to the test that you've linked. The inlined endpoint configuration and the system property both have the same host and port, so it's not clear what the assertions are testing. Does the endpoint override any supplied system properties, or vice versa? https://github.com/aws/aws-sdk-java-v2/blob/fd0a4c2ae7a7ff234cf61d0d63861094c5cbfa11/http-clients/apache-client/src/test/java/software/amazon/awssdk/http/apache/ProxyConfigurationTest.java#L54-L67

[aronsemle]

Ran into this issue as well. We upgraded from V1 to V2 and this broke for a few customers using the props to control the proxy. The work around is to set http.proxyHost and http.proxyPort to point to the HTTPs proxy, but this isn't ideal because if there are other HTTP based connections in our app using the non-secure proxy they will break.

I understand the desire to make things backwards compatible, but I feel like it's a bug/broken and trying to maintain what it's doing now doesn't make sense? That or maybe it's a bit of logic that says "if https.proxy is set, use that, if http.proxy is set and https is not default https to the http"?