FR: Bring back multiple-bucket support

[Mosnar]

We should be able to specify the bucket at the beginning of the URL to support multiple environments.

[Mosnar]

Here's the plan:

• We'll change the URI format to start with the bucket name with the full path.

• We'll drop OBJECT_PREFIX and simply require the full path to the object

• The allowed buckets setting will dictate the IAM rule so you can restrict bucket object access

[Mosnar]

@timkelty After some thinking on this one, I realized there were some issues that may make this worth reconsidering, and I'd like your input.

In order to support multiple buckets, we also need to support multiple object prefixes to prevent arbitrary object access. There isn't a great way as far as I can tell to convert some arbitrary environment variable of buckets to a list of ARNs for the generated IAM policy. We would need to be able to accept something like: staging.assets.site.com/images/*, production.assets.site.com/images/* and convert that into the necessary format.

We support could support just denoting your own role to use in the serverless.yml and then the user could self-manage the allowed assets; HOWEVER, this means each request needs to try to access an object and fail, rather than using the environment variable to validate the request prior to the S3 request. Not sure if this is a big deal or not.

The other option is we use stages to support a single different bucket and object prefix for each environment.

[timkelty]

Does something like this make sense?

• If SOURCE_BUCKET is empty, we assume:
  • user will configure their own role for s3 access in serverless.yml
  • first segment of the path will be the bucket

{SOURCE_BUCKET}/{OBJECT_PREFIX}/{path}?{query}

[timkelty]

FWIW - My main reason for needing this is to handle environment-specific buckets.

If serverless makes it easy enough to deploy a couple envs that are bucket specific…I'm probably ok with that. I'm new to serverless and have avoided this in the past because managing/deploying multiple stacks was a pain.

[ganey]

@timkelty take a look at https://serverless.com/framework/docs/providers/aws/guide/variables/

You can then import bucket name from a file: ${file(./env-vars.yml):image_bucket, "default-s3-bucket"}

Just swap the env-vars.yml out when deploying to different environments

[Mosnar]

Done - see new settings.yml file

[timkelty]

@Mosnar unless I'm mistaken the new settings stuff doesn't really address the original feature request.

The request was to have multiple bucket support on the same deployment, so you wouldn't have to deploy multiple.

That said, serverless/settings.yml makes it easy enough to deploy and manage multiple versions, and supporting multiple buckets is tricky from a permissions perspective, so I say we still cancel/table this for now.

[Mosnar]

Fair nuff

[tyrauber]

I agree with @timkelty. :+1: for this feature request. As long as it doesn't overcomplicate the simple use case / configuration. It would simplify multi-environment, mutli-bucket deployments tremendously allowing a single api gateway and cloudfront distribution to service multiple environments while still segregating media across environment buckets. Additionally, it would enable some really interesting multi-tenant scenarios where a single deployment could service many buckets. Very cool.