artifacts icon indicating copy to clipboard operation
artifacts copied to clipboard

Signature v4-only S3 regions are not supported (us-east-2, eu-central-1, etc.)

Open peterkir opened this issue 10 years ago • 27 comments

When a S3 bucket is created in region "eu-central-1" / Frankfurt it is impossible to use artifacts to upload to it.

peterkir avatar Feb 19 '15 20:02 peterkir

seeing the same issue, getting either:

The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.

or

unsupported protocol scheme ""

Jell avatar Sep 25 '15 12:09 Jell

:+1: same problem

ldonnet avatar Nov 05 '15 14:11 ldonnet

+1 finally use us-west-1 and it worked.

ensonic avatar Dec 07 '15 10:12 ensonic

+1 same problem, tried using eu-central-1.

cmouse avatar Dec 21 '15 13:12 cmouse

The artifacts command line tool works for me with region us-west-2, but Travis CI does not. It appears to ignore the configuration variable addons.s3_region. Works:

ARTIFACTS_REGION=us-west-2 artifacts upload --key=$redacted --secret=$redacted --bucket=foo bar

Travis CI does not work:

err: The bucket you are attempting to access must be addressed using the specified endpoint. Please send all future requests to this endpoint.

https://travis-ci.org/Linuxbrew/linuxbrew/builds/107042959#L4267 My .travis.yml includes:

addons:
  artifacts:
    debug: true
  s3_region: "us-west-2"

sjackman avatar Feb 04 '16 18:02 sjackman

A workaround for Travis CI is to set the environment variable ARTIFACTS_REGION rather than the .travis.yml variable addons.s3_region.

sjackman avatar Feb 04 '16 19:02 sjackman

Just a long shot question, should the .travis.yml variable perhaps be addons.artifacts.s3_region rather than addons.s3_region as the documentation indicates?

sjackman avatar Feb 04 '16 19:02 sjackman

I had the same problem, and can confirm that official travis documentation is INCORRECT on this page:

https://docs.travis-ci.com/user/uploading-artifacts/

The first example shows the setting being addons.s3_region, but the setting is actually addons.artifacts.s3_region

My .travis.yml file now has:

- addons:
  - postgresql: '9.3'
  - artifacts:
    - s3_region: "us-west-2" # defaults to "us-east-1"
    - paths:
      - $(ls tmp/capybara/screenshot_* | tr "\n" ":")

Travis: please fix your documentation! Thanks!

dacoinminster avatar Feb 16 '16 19:02 dacoinminster

@dacoinminster could you please create a PR to https://github.com/travis-ci/docs-travis-ci-com?

brauliobo avatar Mar 12 '16 14:03 brauliobo

It looks like somebody already did (https://github.com/travis-ci/docs-travis-ci-com/issues/524)

dacoinminster avatar Mar 14 '16 15:03 dacoinminster

Adding the ARTIFACTS_REGION environment variable did not change anything for me. Once I changed to a Oregon based bucket, everything worked as expected.

I used the exact same setup for both, once using ARTIFACTS_REGION set to eu-central-1 and once to us-west-2 (so not default!). This is the repository I'm talking about: https://github.com/bbvch/street-light-simulator Although not critical for this project, I would of course like travis to support european buckets.

Deadolus avatar Apr 15 '16 13:04 Deadolus

Hi, everyone,

Sorry for the delay. I pushed an update which I think might fix this issue. Could you try it again?

BanzaiMan avatar Apr 15 '16 17:04 BanzaiMan

Hmmm. It doesn't work just yet. https://travis-ci.org/BanzaiMan/travis_production_test/builds/123420742#L165

ERROR: failed to upload: /home/travis/build/BanzaiMan/travis_production_test/Gemfile
  err: The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256.

https://github.com/mitchellh/goamz/issues/166

BanzaiMan avatar Apr 15 '16 19:04 BanzaiMan

I think https://github.com/goamz/goamz/ might be a better bet going forward (it supports SHA256 https://github.com/mitchellh/goamz/search?utf8=%E2%9C%93&q=sha256).

I tried my hand at transitioning to it; some tweaks are necessary to source to compile, but I could not get tests to pass. See https://travis-ci.org/travis-ci/artifacts/builds/123291124#L219-L222. I read sources to figure out why the error is happening, but my understanding of Go was not enough to push me over the edge.

BanzaiMan avatar Apr 15 '16 23:04 BanzaiMan

Too bad this is still an open issue. Is there any news on this? V4 authentication (http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html) is required for all new AWS regions, so this will be an issue for all new regions. For a German based company I would expect Frankfurt to be supported :laughing:

I think the only way to go now is the official AWS Go SDK at https://github.com/aws/aws-sdk-go which supports V4 out of the box and is updated regularly. I have used this is other projects as well and our production servers upload loads to eu-central-1 buckets using this package.

If I have time I can also take a look at this if needed. No idea what your current code is like but I don't think it should be too much work to change packages.

SebastiaanKlippert avatar Jul 26 '16 14:07 SebastiaanKlippert

Same here. It seems it's still not fixed. Works fine with a bucket in Ireland.

jeschkies avatar Aug 31 '16 20:08 jeschkies

Is there any update on this? I still either get wrong endpoint or use AWS4-HMAC-SHA256 😞

Primajin avatar Apr 12 '17 13:04 Primajin

We've received some similar reports for eu-west-2, eu-east-2 and eu-central regions through support.

iriberri avatar Jul 13 '17 11:07 iriberri

The full list can be seen here: https://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region

The issue is reliance on an old AWS library, https://github.com/mitchellh/goamz.

BanzaiMan avatar Mar 08 '18 21:03 BanzaiMan

Is there any workaround I can use until Travis feels it's a problem that they have support for artifacts: in less than half the AWS S3 regions?

magthe avatar Jul 11 '18 13:07 magthe

I can't believe this issue is still not resolved since 2015, it's almost 2019 out there!

The deploy works fine with eu-west-2 for instance, but artifacts fail with the same infamous error: WARNING: invalid region, defaulting to [secure].

I would gladly use the deploy only, if there's any way to use it when build fails (i.e. uploading test reports to S3).

DenysVuika avatar Sep 20 '18 09:09 DenysVuika

Folks, my team needs this. Can someone at Travis ci please update the aws library?

Incognito avatar Feb 27 '19 18:02 Incognito

Yup. Running into this too. Maybe I just need to install the aws client and do it all myself?

ehuelsmann avatar Mar 16 '19 13:03 ehuelsmann

actually, using pip install awscli --user --upgrade and adding the required aws s3 commands is about 100x faster than using this addon (due to the endless tinkering before finding this bug). Please update documentation to say so.

ehuelsmann avatar Mar 16 '19 16:03 ehuelsmann

eu-central-1 region still doesn't work. The upload process fails with The authorization mechanism you have provided is not supported. Please use AWS4-HMAC-SHA256. message.

remal avatar Jul 13 '20 17:07 remal

Can't believe this problem still exist in 2023 after the 1st reporting in 2015 :(

Myasuka avatar Jan 20 '23 11:01 Myasuka

Hey, if the AWS SDK hasn't been updated in nearly 7 years, is that a security issue at this point?

Incognito avatar Jan 20 '23 18:01 Incognito