Enable use of AWS provided 'default s3 key' and configuration of 'bucket key' for the created s3 bucket

[pgdad]

This is a 'customer cost reduction' issue. A user should be able to optionally to use the the aws/s3 kms key, and to configure it as the default key for the bucket created.

Here are the relevant docs: https://docs.aws.amazon.com/AmazonS3/latest/userguide/default-bucket-encryption.html https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucket-key.html

[mikebroberts]

I strongly agree, and I think it's pretty bad that the CLI is creating (non free) KMS keys without any significant warning. For example, say I'm in a small to medium org, with 100 sub accounts (because of per-developer sub accounts), and we work in 3 regions - if we want to use private modules, which need to be installed to every subaccount (as far as I can tell), that's going to be $300 / month.

Since https://github.com/aws-cloudformation/cloudformation-cli/commit/b8b2c8213e74f76f9f4ddf610d87661faf748e11#diff-f41a0694a67138f5f9447b0a6a03b8fdf2f6e9a2582cdd1ed1c0532b486648e4 , I think SSE-S3 should be sufficient? If so I think it would be prudent to remove the CMK from the template.

[ammokhov]

@pgdad @mikebroberts apologies for the late reply and thank you for creating an issue. cfn submit creates required artifacts for you but if you aren't happy with the layout you could directly use CFN APIs like register-type. This way you could provide custom S3 bucket with preferred SSE.

[mikebroberts]

@ammokhov - I don't think that's a sufficient response. Again, to my reply there is no warning given about the cost implications of using cfn submit AND I don't think SSE-KMS is even necessary any more - if anything SSE-KMS should be the alternative process, not the default .

Also there's no full documentation (that I've found) of using register-type instead, and furthermore performing cleanup of cfn submit 's behavior is a non-trivial task, especially across multiple accounts and/or region.

[benbridts]

I think Mike is right here, the default should not be creating an extra CMK.

However, I do think the default should be SSE-KMS, with the AWS managed CMK. This will remove the cost-component, while still integrating with KMS.

Additionally managed-upload-infrastructure.yaml could be extended with a Parameter so that customers that do need their own CMK (eg, to share artefacts between accounts), can specify a CMK ARN (if nothing is specified, the AWS Managed CMK can be used), that would make it easy for customers to use both cfn submit and their own key policies (assuming cfn does use existing values for parameters when updating that stack).

For customers that already have the managed infrastructure deployed, this change has a small chance of breaking things though. As the removal of the custom CMK will make data in the bucket unreadable. That should not be a problem for places where the artefact is already registered. I can't immediate imagine cases where the artefact would be in the bucket without being registered, but if you edit the bucket policy or key policy everything is possible.

@mikebroberts Not using the managed infra is what I would do for cross account deployments though.

1. Create the zip file with cfn submit --dry-run
2. Upload it to your own bucket, with a bucket policy that allows reads from the whole organization (or relevant accounts)
3. Deploy a StackSet with an AWS::IAM::Role, AWS::CloudFormation::ResourceVersion (pointing to the file in S3), and AWS::CloudFormation::ResourceDefaultVersion

[mikebroberts]

However, I do think the default should be SSE-KMS, with the AWS managed CMK. This will remove the cost-component, while still integrating with KMS.

Do you mean SSE-S3 here? Or is there a free version of SSE-KMS that I'm ignorant about? :)

@mikebroberts Not using the managed infra is what I would do for cross account deployments though.

Yep, that's the way to do it right now. I've made a request with the CloudFormation team to build in cross-account and cross-region support for the registry in general (like SAR has, I think, at least the public version), but in the short term you're right.

3. Deploy a StackSet with an AWS::IAM::Role, AWS::CloudFormation::ResourceVersion (pointing to the file in S3), and AWS::CloudFormation::ResourceDefaultVersion

(Getting slightly off topic...) I took a very quick look at the CloudFormation::Resource* types - is it true that you need to maintain a ResourceVersion instance for every deployed version of the module? What happens if you delete a ResourceVersion instance?

[benbridts]

Do you mean SSE-S3 here? Or is there a free version of SSE-KMS that I'm ignorant about? :)

There is, from the docs here and here: "When you use server-side encryption with AWS KMS (SSE-KMS), you can use the default AWS managed CMK, or you can specify a customer managed CMK that you have already created." and "You do not pay a monthly fee for AWS managed CMKs. They can be subject to fees for use in excess of the free tier, but some AWS services cover these costs for you"

is it true that you need to maintain a ResourceVersion instance for every deployed version of the module

If you put the ResourceVersion and ResourceDefaultVersion in the same template, you can use the same logicalId and have CloudFormation delete the older versions according to its UpdateReplacePolicy. Changing the S3 URL replaces the physical resource

[mikebroberts]

@benbridts - I stand corrected - thanks!