aws s3 cp --recursive returns success when prefix (dir) doesn't exist

[bgdnlp]

Describe the bug

aws s3 cp s3://my-bucket/no/such/directory/ --recursive

will always exit with error code 0 (success).

I'm not entirely sure if this should be considered a bug, but it was a surprise for me. Version

Regression Issue

• [ ] Select this option if this issue appears to be a regression.

Expected Behavior

aws s3 cp will return 1 if the object doesn't exist. I expected --recursive to do the same.

aws s3 ls --recursive will also return 1 on a prefix that doesn't exist.

Current Behavior

returns 0 instead of 1

Reproduction Steps

aws --profile dprod s3 cp s3://noaa-goes16/ABI-L1b-RadC/2020/no/suc/dir/ . --recursive ; echo $?

Possible Solution

No response

Additional Information/Context

This is part of a script that was supposed to fail if it couldn't download a file. Can be worked around by running aws s3 ls first, but that's a reason to consider it a bug maybe?

CLI version used

2.28.24

Environment details (OS name and version, etc.)

Mac/Linux

[RyanFitzSimmonsAK]

Hi @bgdnlp, thanks for reaching out and for your patience. You are correct that this is not a bug. What's happening here is that s3 cp uses a ListObjectsV2 operation to get the list of objects with the prefix you specified. ListObjectsV2 does not demand that the prefix exists; it's just a filter. So it doesn't throw an error, it just returns an empty list, and s3 cp then copies any files returned by that operation, which happen to be none in this case.

I can definitely see how this would be surprising, given that we don't have an example that shows this. aws s3 ls has similar behavior and has an example (#3) that demonstrates this a bit better. Please let me know if you have any follow-up questions.

[bgdnlp]

Up to you how you solve the report. I my view simply adding a flag to process more than one file should not change how the files are processed.

The aws s3 ls example is not relevant because

• it doesn't talk about the returned exit code, just about the printed output
• if it did talk about the exit code it would underline the inconsistency I'm talking about since aws ls --recursive returns 1 as expected, unlike aws cp --recursive.

I can't say what the right answer is. In my experience similar commands return 1, as I would expect. For example copying a directory, cp -R source destination.