Folder sharing - Replace S3 bucket policies for S3 Access Points

[dlpzx]

Current implementation of folder sharing is based on S3 bucket policies in which we grant permissions to certain principals to certain prefixes inside the S3 Bucket. When a new folder is added to the share request, the dataset CloudFormation stacks is updated and updates the dataset S3 Bucket policy.

For the scenario in which multiple folders are shared with multiple different teams, we can reach the limit of 20KB size for bucket policies.

I would like a solution that scales for any number of folders shared and requester teams. In addition, I would like to keep dataset deployment separated from shares.

P.S. Don't attach files. Please, prefer add code snippets directly in the message body.