serverless icon indicating copy to clipboard operation
serverless copied to clipboard
verified publisher icon serverless

Attaching AWS::S3::BucketPolicy hangs forever on CREATE_IN_PROGRESS

Open tenpaiyomi opened this issue 2 years ago • 8 comments

Are you certain it's a bug?

  • [X] Yes, it looks like a bug

Is the issue caused by a plugin?

  • [X] It is not a plugin issue

Are you using the latest v3 release?

  • [X] Yes, I'm using the latest v3 release

Is there an existing issue for this?

  • [X] I have searched existing issues, it hasn't been reported yet

Issue description

When attempting to create an AWS::S3::Bucket and then attach an AWS::S3::BucketPolicy to it, the AWS::S3::BucketPolicy will hang forever in the CREATE_IN_PROGRESS state. To ensure I was not doing something wrong here, I modified my CloudFormation template in the designer and ensured it was valid, which was confirmed. I then attempted to roll out this same stack with CloudFormation directly, successfully.

For reference, here is the applicable portion of the cloudformation.yml / serverless.yml

serverless.yml

  BuildStorageBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: ${self:provider.environment.DOWNLOADS_STORAGE_BUCKET_NAME}
      OwnershipControls:
        Rules:
          - ObjectOwnership: ObjectWriter
      PublicAccessBlockConfiguration:
        BlockPublicAcls: false
        BlockPublicPolicy: false
        IgnorePublicAcls: false
        RestrictPublicBuckets: false
  BuildStorageBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
        Bucket: !Ref BuildStorageBucket
        PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Sid: AllowPublicReadAccess
                Effect: Allow
                Principal: '*'
                Resource: !Sub 'arn:aws:s3:::${BuildStorageBucket}/*'
                Action:
                  - 's3:GetObject'
    DependsOn:
      - BuildStorageBucket

cloudformation.yml

  BuildStorageBucket:
    Type: 'AWS::S3::Bucket'
    Properties:
      BucketName: dev-downloads-storage
      OwnershipControls:
        Rules:
          - ObjectOwnership: ObjectWriter
      PublicAccessBlockConfiguration:
        BlockPublicAcls: false
        BlockPublicPolicy: false
        IgnorePublicAcls: false
        RestrictPublicBuckets: false
    Metadata:
      'AWS::CloudFormation::Designer':
        id: dbdc4cce-6b8d-4982-b2c4-1bc7cbe89300
  BuildStorageBucketPolicy:
    Type: 'AWS::S3::BucketPolicy'
    Properties:
        Bucket: !Ref BuildStorageBucket
        PolicyDocument:
            Version: 2012-10-17
            Statement:
              - Sid: AllowPublicReadAccess
                Effect: Allow
                Principal: '*'
                Resource: !Sub 'arn:aws:s3:::${BuildStorageBucket}/*'
                Action:
                  - 's3:GetObject'
    Metadata:
      'AWS::CloudFormation::Designer':
        id: cbbd0a28-7974-45d9-b6c3-31f161bfcc82
    DependsOn:
      - BuildStorageBucket

Noting that the only difference between these two are the Metadata blocks and the bucket name replacement.

I am unsure of if this is explicitly an aws thing or a serverless thing, but it does work from the AWS Console so I figured bringing it up here is worthwhile. If I remove the AWS::S3::BucketPolicy from the serverless.yml, the stack and updates deploy without issue.

Service configuration (serverless.yml) content

N/A

Command name and used flags

sls deploy --stage dev --verbose

Command output

Deploying to stage dev
global › waiting
storage-authorizer › waiting
func-lambdas › waiting
apollo-server › waiting
serverOutputs › waiting
clientOutputs › waiting
global › deploying
global › Running "serverless deploy --stage dev"
global › Running "serverless" from node_modules
global › Deploying global to stage dev (us-east-2)
global › Uploading CloudFormation file to S3
global › Uploading State file to S3
global › Creating new change set
global › Waiting for new change set to be created
global › Change Set did not reach desired state, retrying
global › Executing created change set
global › UPDATE_IN_PROGRESS - AWS::CloudFormation::Stack - global-dev
global › CREATE_IN_PROGRESS - AWS::DynamoDB::Table - connectionsTable
global ›   CREATE_IN_PROGRESS - AWS::DynamoDB::Table - MembershipsTable
global ›   CREATE_IN_PROGRESS - AWS::S3::Bucket - SessionStorageBucket
global ›   CREATE_IN_PROGRESS - AWS::DynamoDB::Table - subscriptionsTable
global ›   CREATE_IN_PROGRESS - AWS::S3::Bucket - BuildStorageBucket
global › CREATE_IN_PROGRESS - AWS::DynamoDB::Table - ConfigurationsTable
global ›   CREATE_IN_PROGRESS - AWS::DynamoDB::Table - WorkspaceTable
global ›   CREATE_IN_PROGRESS - AWS::Cognito::UserPool - CognitoUserPool
global ›   CREATE_IN_PROGRESS - AWS::S3::Bucket - BuildStorageBucket
global ›   CREATE_IN_PROGRESS - AWS::S3::Bucket - SessionStorageBucket
global ›   CREATE_IN_PROGRESS - AWS::DynamoDB::Table - connectionsTable
global ›   CREATE_IN_PROGRESS - AWS::DynamoDB::Table - MembershipsTable
global ›   CREATE_IN_PROGRESS - AWS::DynamoDB::Table - subscriptionsTable
global ›   CREATE_IN_PROGRESS - AWS::DynamoDB::Table - WorkspaceTable
global ›   CREATE_IN_PROGRESS - AWS::DynamoDB::Table - ConfigurationsTable
global › CREATE_IN_PROGRESS - AWS::Cognito::UserPool - CognitoUserPool
global ›   CREATE_COMPLETE - AWS::Cognito::UserPool - CognitoUserPool
global › CREATE_IN_PROGRESS - AWS::Cognito::UserPoolClient - CognitoUserPoolClient
global ›   CREATE_IN_PROGRESS - AWS::Cognito::UserPoolClient - CognitoUserPoolClient
global ›   CREATE_COMPLETE - AWS::Cognito::UserPoolClient - CognitoUserPoolClient
global ›   CREATE_IN_PROGRESS - AWS::Cognito::IdentityPool - CognitoIdentityPool
global ›   CREATE_IN_PROGRESS - AWS::Cognito::IdentityPool - CognitoIdentityPool
global › CREATE_COMPLETE - AWS::Cognito::IdentityPool - CognitoIdentityPool
global › CREATE_COMPLETE - AWS::DynamoDB::Table - connectionsTable
global ›   CREATE_COMPLETE - AWS::DynamoDB::Table - ConfigurationsTable
global › CREATE_COMPLETE - AWS::S3::Bucket - BuildStorageBucket
global ›   CREATE_COMPLETE - AWS::S3::Bucket - SessionStorageBucket
global › CREATE_COMPLETE - AWS::DynamoDB::Table - MembershipsTable
global ›   CREATE_COMPLETE - AWS::DynamoDB::Table - WorkspaceTable
global ›   CREATE_COMPLETE - AWS::DynamoDB::Table - subscriptionsTable
global ›   CREATE_IN_PROGRESS - AWS::S3::BucketPolicy - BuildStorageBucketPolicy
global › CREATE_IN_PROGRESS - AWS::S3::BucketPolicy - BuildStorageBucketPolicy

    ⠇  global › deploying › 829s
       storage-authorizer › waiting
       func-lambdas › waiting
       apollo-server › waiting
       serverOutputs › waiting
       clientOutputs › waiting

Environment information

Framework Core: 3.36.0
Plugin: 7.1.0
SDK: 4.4.0

tenpaiyomi avatar Nov 15 '23 19:11 tenpaiyomi

Hi, @tenpaiyomi I had the same problem in my case the problem was in the Version of the BucketPolicy. I needed to add quotes to it.

BuildStorageBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
        Bucket: !Ref BuildStorageBucket
        PolicyDocument:
            Version: '2012-10-17' # IMPORTANT ---> See this is now quoted
            Statement:
              - Sid: AllowPublicReadAccess
                Effect: Allow
                Principal: '*'
                Resource: !Sub 'arn:aws:s3:::${BuildStorageBucket}/*'
                Action:
                  - 's3:GetObject'

If we don't quote the Version. Then, the Serverless framework thinks this is a date, generating the following in the CloudFormation template: "Version": "2012-10-17T00:00:00.000Z" instead of just: "Version": "2012-10-17".

This error makes the Cloudformation service stay forever in the CREATE_IN_PROGRESS state for the resource. Even though it should return a validation error.

jorovipe97 avatar Nov 29 '23 16:11 jorovipe97

I still have this issue and have tried many different slightly different versions but can't get to the bottom of it as there is no validation error message.

Mine looks like this:

SchemasBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref SchemasBucket
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Sid: AllowCloudfrontReadAccess
            Action:
              - 's3:GetObject'
            Effect: Allow
            Principal:
              Service: cloudfront.amazonaws.com
            Resource: !GetAtt SchemasBucket.Arn
            Condition:
              StringEquals:
                'AWS:SourceArn': !Sub arn:aws:cloudfront::${AWS::AccountId}:distribution/${Distribution.Id}

That's the latest version I tested, but it keeps getting stuck. I already tried about a dozen different versions with slight changes in quoting, ordering, using a string instead of an array for Action, and other subtle changes, but nothing seems to work.

Any suggestions on what else I could try are welcome.

EDIT

It was a silly mistake on my end... I was missing the /* at the end of the resource, so it was trying to assign the policy to the whole bucket instead of the objects within the bucket. That fixed it.

astro-nahuel avatar Mar 02 '24 19:03 astro-nahuel

I am experiencing this issue with Serverless Framework, my deployment just hangs. This is the code:


 SamplePolicy:
      Type: AWS::S3::BucketPolicy
      DependsOn: SampleBucket
      Properties:
        Bucket: !Ref SampleBucket
        PolicyDocument:
          Version: '2012-10-17'
          Statement:
            - Sid: "GetBucket"
              Action:
                - s3:GetBucketAcl
                - s3:GetBucketPolicy
              Effect: Allow
              Principal: "*"
              Resource: "*"
            - Sid: "PutObjects"
              Action:
                - s3:PutObject
              Effect: Allow
              Principal: "*"
              Resource: "*"

ststzuhlke avatar Apr 30 '24 23:04 ststzuhlke

Hi @ststzuhlke. It seems that the problem is in the Resource: attributes. You cannot just use an asterisk to indicate "Any Object" or "Any Bucket".

SamplePolicy:
  Type: AWS::S3::BucketPolicy
  DependsOn: SampleBucket
  Properties:
    Bucket: !Ref SampleBucket
    PolicyDocument:
      Version: '2012-10-17'
      Statement:
        - Sid: "GetBucket"
          Action:
            - s3:GetBucketAcl
            - s3:GetBucketPolicy
          Effect: Allow
          Principal: "*"
          Resource: !Sub 'arn:aws:s3:::${SampleBucket}' # s3:GetBucketAcl and s3:GetBucketPolicy IAM actions apply to "bucket" resource types
        - Sid: "PutObjects"
          Action:
            - s3:PutObject
          Effect: Allow
          Principal: "*"
          Resource: !Sub 'arn:aws:s3:::${SampleBucket}/*' # s3:PutObject IAM action apply to "objects in the bucket" resource type. That's why we need the /* at the end.

Depending on the IAM action you must specify a correct resource ARN. To see a detailled list of all resource ARNs for all IAM S3 actions you can look at this page in the AWS Docs.

Note I did not tested this myself in CloudFormation but probably that is the problem.

jorovipe97 avatar May 02 '24 15:05 jorovipe97

Had the exact same issue and tried to figure out why it was timing out. I added the missing quotations and it worked.

tcope25 avatar May 15 '24 18:05 tcope25

I encountered similar issue, but in my case Principal was referring to a role in another aws account. The stack was hanging because the role in the other account was not yet created. The fix in my case was to create the role in the other account beforehand.

uxon123 avatar Aug 08 '24 12:08 uxon123