NullPointerException while invoking the "writeHttp2RequestMetrics"

[guysung]

Describe the bug

Race condition in AWS SDK 2.31.78 where HTTP/2 stream gets cleaned up before metrics collection completes, causing NPE in NettyRequestMetrics.writeHttp2RequestMetrics().

Please see the following stack trace for more details.

Stack Trace: java.lang.NullPointerException: Cannot invoke \"io.netty.handler.codec.http2.Http2Stream.getProperty(io.netty.handler.codec.http2.Http2Connection$PropertyKey)\" because \"stream\" is null at io.netty.handler.codec.http2.DefaultHttp2LocalFlowController.state(DefaultHttp2LocalFlowController.java:289) at io.netty.handler.codec.http2.DefaultHttp2LocalFlowController.windowSize(DefaultHttp2LocalFlowController.java:157) at software.amazon.awssdk.http.nio.netty.internal.NettyRequestMetrics.writeHttp2RequestMetrics(NettyRequestMetrics.java:82) at software.amazon.awssdk.http.nio.netty.internal.NettyRequestMetrics.lambda$publishHttp2StreamMetrics$0(NettyRequestMetrics.java:62) at java.base/java.util.Optional.ifPresent(Optional.java:178) at software.amazon.awssdk.http.nio.netty.internal.NettyRequestMetrics.publishHttp2StreamMetrics(NettyRequestMetrics.java:61) at software.amazon.awssdk.http.nio.netty.internal.NettyRequestExecutor.lambda$writeRequest$13(NettyRequestExecutor.java:250)

It appears that null-check for stream is needed in NettyRequestMetrics.writeHttp2RequestMetrics()

Regression Issue

• [ ] Select this option if this issue appears to be a regression.

Expected Behavior

No NullPointerException

Current Behavior

NullPointerException

Reproduction Steps

The error occurred many times while making AWS Transcriber requests on Oct 25th when recovering from the AWS east-1 outage. It's hard to reproduce after then.

Possible Solution

Adding the null-check for the stream

private static void writeHttp2RequestMetrics(MetricCollector metricCollector, Channel channel, Http2Connection http2Connection) { int streamId = channel.attr(ChannelAttributeKey.HTTP2_FRAME_STREAM).get().id();

Http2Stream stream = http2Connection.stream(streamId);
if (stream != null) {
    metricCollector.reportMetric(Http2Metric.LOCAL_STREAM_WINDOW_SIZE_IN_BYTES,
                                 http2Connection.local().flowController().windowSize(stream));
    metricCollector.reportMetric(Http2Metric.REMOTE_STREAM_WINDOW_SIZE_IN_BYTES,
                                 http2Connection.remote().flowController().windowSize(stream));
}

}

Additional Information/Context

No response

AWS Java SDK version used

2.31.78

JDK version used

17

Operating System and version

AWS EC2 c5.xlarge

[bhoradc]

Hi @guysung

Thank you for reporting the issue. While I understand you cannot easily reproduce the race condition, I see a valid reason for the null check. That's becausehttp2Connection.stream(streamId) can return null when the stream has been cleaned up, but the code doesn't check for this before calling flowController.windowSize(stream). We'll look into this further.

Minimal reproducible code sample

package org.example;

import io.netty.handler.codec.http2.DefaultHttp2Connection;
import io.netty.handler.codec.http2.DefaultHttp2LocalFlowController;
import io.netty.handler.codec.http2.Http2Stream;

public class Main {
    public static void main(String[] args) {
            DefaultHttp2Connection connection = new DefaultHttp2Connection(false);
            DefaultHttp2LocalFlowController flowController = new DefaultHttp2LocalFlowController(connection);
            Http2Stream nullStream = null;
            flowController.windowSize(nullStream);
    }
}

Regards, Chaitanya