aws
v2 Transfer Manager preview throws NullPointerException
Describe the bug
When downloading a file, we are seeing NullPointerExceptions thrown:a
2022-08-30 18:25:53.781 Exception in thread "Thread-2" java.lang.NullPointerException
2022-08-30 18:25:53.781 at java.nio.ByteBuffer.wrap(ByteBuffer.java:396)
2022-08-30 18:25:53.781 at software.amazon.awssdk.crt.s3.S3MetaRequestResponseHandlerNativeAdapter.onResponseBody(S3MetaRequestResponseHandlerNativeAdapter.java:15)
Expected Behavior
We did not expect Null Pointer Exceptions.
Current Behavior
Downloading a file throws an NPE
Reproduction Steps
Unable to consistently reproduce. Seeing it on larger files being downloaded, like ~ 2 GB.
Possible Solution
S3 response body may be empty?
Additional Information/Context
No response
AWS Java SDK version used
2.17.264-PREVIEW
JDK version used
8
Operating System and version
Linux
Also shoul add we were setting targetBandwidth and partSize to:
2Gbps and 256MB, respectively
Hello @danbills ,
Thank you very much for your submission. I need more information to investigate this issue.
- Could you please share a reproducible code sample.
- Could you share the CRT Trace logs to help investigating the origin of this behavior? You can enable CRT Trace logs by adding the following to your code:
Log.initLoggingToFile(Log.LogLevel.Trace, <filename>);
Note : make sure to redact any sensitive information before sharing the file here.
Best,
Yasmine
Hi @yasminetalby , thanks very much for your prompt attention to this!
Could you please share a reproducible code sample.
I'm afraid we are unable to reproduce this consistently, just noticing a trend of it occurring on larger files. The responses from S3 seem to be where it's happening (onResponseBody) so my guess is we would have to mock some kind of unusual S3 response to reproduce.
Could you share the CRT Trace logs to help investigating the origin of this behavior?
Sure, will follow up with that info.
Hello @danbills ,
The CRT logs will allow for further investigation of the behavior. Thank you for your collaboration!
Best,
Yasmine
Error is at 13:41:58.032
best viewed with less -r as it is ANSI-encoded.
@yasminetalby
Hello @danbills ,
Thank you very much for the documentation provided and for bringing this issue to our attention. I was wondering if you could give me more context on your use of the S3 transfer manager? Could you provide details on your client configuration? What credential provider when attempting the download?
Thank you very much for your collaboration! Sincerely,
Yasmine
Hello @danbills ,
Update: After investigation, I have reached out to the CRT team as this issue seems to be originated from the CRT client which is used by the S3TransferManager. They have confirmed that this is a known issue on their end and that it will be resolved before GA. (Note that both the S3TransferManager and the CRT Client are currently in PREVIEW)
The current mitigation for this behavior is to increase heap size. Let me know if you have any further questions or concern.
Thank you very much for submitting this issue and for your collaboration.
Best regards,
Yasmine
@yasminetalby
That's good info! Thanks for the update. I suppose we should keep this open until we can validate w/ some kind of update from CRT client.
Yes! I will keep this open to post an update once we get a resolution update from the CRT team and for other users to be aware of this temporary behavior using S3TransferManager if that is ok with you.
Thank you very much for bringing this up to our team! Best,
Yasmine
Hi @danbills, the change has been released. Could you try with the latest version (2.19.0 as of now)
@zoewangg we will be testing the latest version in the coming months. Is it out of preview now? Noticed the new version omits that
Hello @danbills,
Thank you very much, we really appreciate your interest and contribution. Yes, the Transfer Manager is now out of preview (see)
Best,
Yasmine