S3EventNotificationRecord does not store information about lifecycleEventData

[sfc-gh-hchandrasekaran]

Describe the bug

I get an event from s3 after an object is moved from STANDARD to DEEP_ARCHIVE storage class. Since this transition happens due to a lifecycle rule in the bucket this event is of the type 'LifecycleTransition'. The AWS documentation mentions that lifecycleTransition event messages contain the following piece of information.

"lifecycleEventData":{
    "transitionEventData": {
        "destinationStorageClass": the destination storage class for the object
    }
}

I do get this information in the message from the SQS but when I use the SDK to transform this message to an S3EventNotification, this information is lost. The S3EventNotificationRecord object does not seem to have any attribute to store this lifecycleEventData information . Is this a bug that needs to be fixed or is there something I am missing out on?

Expected Behavior

Records in S3EventNotification should consist of 'lifecycleEventData' information.

Current Behavior

S3EventNotification does not include lifecycleEventData information. These are the attributes it currently tracks in the class.

public static class S3EventNotificationRecord {

        private final String awsRegion;
        private final String eventName;
        private final String eventSource;
        private DateTime eventTime;
        private final String eventVersion;
        private final RequestParametersEntity requestParameters;
        private final ResponseElementsEntity responseElements;
        private final S3Entity s3;
        private final UserIdentityEntity userIdentity;
        private final GlacierEventDataEntity glacierEventData;
}

Reproduction Steps

Add an object to an s3 bucket. Set a lifecycle rule that moves objects from STANDARD to DEEP_ARCHIVE storage class within 0 days. Wait for the event to trigger (This takes approximately 24-48hours) and add log statements in Java code to capture this event message. You need to have a setup where an SQS queue is configured to capture events from the bucket. SQS ARN needs to be added to bucket event notification configuration in bucket properties, so that this bucket sends notifications to the SQS being used in your Java application.

Alternatively to avoid waiting you can mock the lifecycleTransition event and send it to your SQS queue within the Java code.

Possible Solution

Include attribute 'lifecycleEventData' in class S3EventNotificationRecord which is within S3EventNotification class to

Additional Information/Context

This information is only sent when s3 transitions object storage classes because of a lifecycle rule being triggered. Manually changing the storage class of an object sends a objectCreated:copy event.

AWS Java SDK version used

1.12

JDK version used

11

Operating System and version

centos 7

[debora-ito]

Hi @sfc-gh-hchandrasekaran thank you for reaching out.

Could you enable the verbose wirelogs with the receiving event? I'd like to see how the request looks like - https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/logging-slf4j.html#sdk-java-logging-verbose

Please be sure to remove any sensitive data the logs might contain.

[sfc-gh-hchandrasekaran]

Thanks for the response.

It would be great if you could please create a mini-app on your end and set notifications and log it. LifecycleTransition events take a lot of time to come in since Amazon evaluates lifecycle rules on a bucket once a day. Transitioning an object to glacier or deep archive takes time even with the transition days set to 0 while creating the lifecycle rule. Hence if you could internally do this it would be much faster. I can see in the aws-java-sdk-s3 in the S3EventNotification.S3EventNotificationRecord class that there is no attribute which currently tracks "lifecycleEventData" that comes in with a lifecycleTransition event.

I am attaching the message I receive before parsing using S3EventNotification.parseJson(). You can clearly see that lifecycleEventData exists. However, after the parsing the records of the type S3EventNotificationRecord contained in the S3EventNotification don't contain the lifecycleEventData.

Thanks for considering my request

On Fri, Jun 17, 2022 at 3:49 PM Debora N. Ito @.***> wrote:

Hi @sfc-gh-hchandrasekaran https://github.com/sfc-gh-hchandrasekaran thank you for reaching out.

Could you enable the verbose wirelogs with the receiving event? I'd like to see how the request looks like - https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/logging-slf4j.html#sdk-java-logging-verbose

Please be sure to remove any sensitive data the logs might contain.

— Reply to this email directly, view it on GitHub https://github.com/aws/aws-sdk-java/issues/2783#issuecomment-1159288467, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZJUIPPCAIIV7UNKO6PT2VLVPT6JLANCNFSM5Y5EOW7Q . You are receiving this because you were mentioned.Message ID: @.***>

[jason-weddington]

Hi @sfc-gh-hchandrasekaran, SDK manager here. We're working with S3 on this now and will provide an update when it's ready for testing.

[sfc-gh-hchandrasekaran]

Great, thanks for the update.

On Thu, Jun 23, 2022 at 8:14 AM Jason Weddington @.***> wrote:

Hi @sfc-gh-hchandrasekaran https://github.com/sfc-gh-hchandrasekaran, SDK manager here. We're working with S3 on this now and will provide an update when it's ready for testing.

— Reply to this email directly, view it on GitHub https://github.com/aws/aws-sdk-java/issues/2783#issuecomment-1164539608, or unsubscribe https://github.com/notifications/unsubscribe-auth/AZJUIPPU74CNDGW3RJ6OGNDVQR5LVANCNFSM5Y5EOW7Q . You are receiving this because you were mentioned.Message ID: @.***>

[debora-ito]

@sfc-gh-hchandrasekaran I believe support for this was added in Java SDK version 1.12.266. Please check it out and let us know if you have any feedback.

[github-actions[bot]]

It looks like this issue has not been active for more than five days. In the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please add a comment to prevent automatic closure, or if the issue is already closed please feel free to reopen it.