Where is it getting credentials from?
I know this isn't exactly a supported app anymore, but it looked like it would still do the trick for a quick and dirt backfill of CT into Elastic.
However, I'm stuck at
2019/11/07 01:45:02.263991 ctbeat.go:266: ERR Unable to list objects in bucket: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
status code: 403, request id: A631EF91513E487C
2019/11/07 01:45:02.264022 beat.go:264: INFO cloudtrailbeat cleanup
2019/11/07 01:45:02.264031 beat.go:316: CRIT Exiting: Error backfilling logs: Failed to list bucket objects: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
status code: 403, request id: A631EF91513E487C
Exiting: Error backfilling logs: Failed to list bucket objects: InvalidAccessKeyId: The AWS Access Key Id you provided does not exist in our records.
status code: 403, request id: A631EF91513E487C
Access to the bucket(s) is/are granted through an assume-role and I've tested that works.
I have tried putting the credentials into environment variables and in debugging desperation do something like this:
export AWS_SECRET_ACCESS_KEY=bla
export AWS_ACCESS_KEY_ID=foo
export AWS_SECURITY_TOKEN=reallylongfubar
aws s3 ls
env
/root/cloudtrailbeat -e -c /config/cloudtrailbeat.yml -d "*" -b ${BUCKET}
This is run from inside a Docker container which has ~/.aws mapped into /root/.aws
With the environment variables set, the aws s3 ls can list the bucket (and I also successfully can do aws s3 cp s3:bucket . --recursive it's just too long to show here). The dummy env command also shows the environment variables set. But the executable doesn't pick them up it seems.
In the source (and I am by no means fluent in Golang) I see reference to the AWS credentials provider, but no hint as to what would need to go into the config YAML.
I then even copied the environment variables into the [default] section in ~/.aws/credentials but all that gets me is
2019/11/07 01:54:16.749157 ctbeat.go:266: ERR Unable to list objects in bucket: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
2019/11/07 01:54:16.749225 beat.go:264: INFO cloudtrailbeat cleanup
2019/11/07 01:54:16.749237 beat.go:316: CRIT Exiting: Error backfilling logs: Failed to list bucket objects: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
Exiting: Error backfilling logs: Failed to list bucket objects: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors