wal-e icon indicating copy to clipboard operation
wal-e copied to clipboard

S3ResponseError: 403 Forbidden using assumed role credentials

Open mylons opened this issue 8 years ago • 1 comments

assume those variables are filled out with valid values, wal-e fails. if i use the same environment variables, and aws s3 ls <bucket> it works fine. I can aws s3 ls all the sub objects of the bucket too.

version: 1.0.3

~$ WALE_S3_PREFIX="s3://some_bucket" AWS_REGION="" AWS_SESSION_TOKEN="" AWS_ACCESS_KEY_ID="" AWS_SECRET_ACCESS_KEY="" wal-e backup-list
wal_e.main   INFO     MSG: starting WAL-E
        DETAIL: The subcommand is "backup-list".
        STRUCTURED: time=2017-05-17T00:07:05.815298-00 pid=5499
name	last_modified	expanded_size_bytes	wal_segment_backup_start	wal_segment_offset_backup_start	wal_segment_backup_stop	wal_segment_offset_backup_stop
wal_e.main   CRITICAL MSG: An unprocessed exception has avoided all error handling
        DETAIL: Traceback (most recent call last):
          File "/opt/virtualenvs/airflow/lib/python3.5/site-packages/wal_e/cmd.py", line 602, in main
            backup_cxt.backup_list(query=args.QUERY, detail=args.detail)
          File "/opt/virtualenvs/airflow/lib/python3.5/site-packages/wal_e/operator/backup.py", line 66, in backup_list
            for bi in bl_iter:
          File "/opt/virtualenvs/airflow/lib/python3.5/site-packages/wal_e/worker/base.py", line 153, in __iter__
            for key in self._backup_list(self.layout.basebackups()):
          File "/opt/virtualenvs/airflow/lib/python3.5/site-packages/boto/s3/bucketlistresultset.py", line 34, in bucket_lister
            encoding_type=encoding_type)
          File "/opt/virtualenvs/airflow/lib/python3.5/site-packages/boto/s3/bucket.py", line 473, in get_all_keys
            '', headers, **params)
          File "/opt/virtualenvs/airflow/lib/python3.5/site-packages/boto/s3/bucket.py", line 411, in _get_all
            response.status, response.reason, body)
        boto.exception.S3ResponseError: S3ResponseError: 403 Forbidden
        <?xml version="1.0" encoding="UTF-8"?>
        <Error><Code>InvalidAccessKeyId</Code><Message>The AWS Access Key Id you provided does not exist in our records.</Message><AWSAccessKeyId></AWSAccessKeyId><RequestId></RequestId><HostId></HostId></Error>

        STRUCTURED: time=2017-05-17T00:07:05.878287-00 pid=5499

mylons avatar May 17 '17 00:05 mylons

I've discovered the source of the issue. wal-e is relying on AWS_SECURITY_TOKEN instead of AWS_SESSION_TOKEN: https://github.com/wal-e/wal-e/blob/master/wal_e/cmd.py#L412

wal-e will work if I use AWS_SECURITY_TOKEN, but the new aws docs, and awscli docs say to use AWS_SESSION_TOKEN

mylons avatar May 17 '17 00:05 mylons