aws-cli icon indicating copy to clipboard operation
aws-cli copied to clipboard
verified publisher icon aws

datasync create-location-object-storage requires --agent-arns despite documentation stating it's optional for agentless Enhanced mode transfers

Open ThomasMorrow-SOI opened this issue 3 months ago • 4 comments

Describe the bug

When attempting to create an Object Storage location for Google Cloud Storage without agents (for use with Enhanced mode), the create-location-object-storage command fails and demands the --agent-arns argument, even though the official documentation states:

"A DataSync agent is only required when using Basic mode tasks. If you are using Enhanced mode to transfer between Google Cloud Storage (GCS) and Amazon S3, then no agent is required."

And the CLI reference describes --agent-arns as:

"(Optional) Specifies the Amazon Resource Names (ARNs) of the DataSync agents that can connect with your object storage system. If you are setting up an agentless cross-cloud transfer, you do not need to specify a value for this parameter."

Regression Issue

  • [ ] Select this option if this issue appears to be a regression.

Expected Behavior

The command should succeed without specifying --agent-arns when creating a location for agentless cross-cloud transfers (GCS to S3).

Current Behavior

The command fails with an error requiring --agent-arns to be specified.

aws: error: the following arguments are required: --agent-arns

Reproduction Steps

aws datasync create-location-object-storage --server-hostname storage.googleapis.com --server-protocol HTTPS --server-port 443 --bucket-name my-gcs-bucket --subdirectory /my-folder --access-key GOOG1EXXXXXXXXXXXXXXXXXXXX --secret-key XXXXXXXXXXXXXXXXXXXXXXXXXXXX --region us-east-1

Possible Solution

No response

Additional Information/Context

This appears to be related to the recent launch of Enhanced mode support for cross-cloud transfers (announced May 29, 2025). The CLI validation logic may not have been updated to reflect that --agent-arns is optional for agentless transfers.

CLI version used

aws-cli/2.27.0 Python/3.14.0 Linux/6.17.1-300.fc43.x86_64 source/x86_64.fedora.43

Environment details (OS name and version, etc.)

OS: Fedora 43 Python Version: 3.14.0

ThomasMorrow-SOI avatar Oct 14 '25 13:10 ThomasMorrow-SOI

Hello @ThomasMorrow-SOI, thanks for reaching out. Could you please add --debug at the end of the command for logging. Then please relay to us all of the Response headers such as Response body and Response headers and we would like to see the error stack trace too. Although, please do redact any sensitive and security information. Thank you.

adev-code avatar Oct 15 '25 16:10 adev-code

aws datasync create-location-object-storage
--server-hostname storage.googleapis.com
--server-protocol HTTPS
--server-port 443
--bucket-name MYBUCKET
--subdirectory MYDIR
--access-key $GCS_ACCESS_KEY_ID
--secret-key $GCS_SECRET_ACCESS_KEY
--region us-east-1
--debug 2025-10-15 12:56:59,715 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.27.0 Python/3.14.0 Linux/6.17.1-300.fc43.x86_64 source/x86_64.fedora.43 2025-10-15 12:56:59,715 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['datasync', 'create-location-object-storage', '--server-hostname', 'storage.googleapis.com', '--server-protocol', 'HTTPS', '--server-port', '443', '--bucket-name', 'MYBUCKET', '--subdirectory', 'MYDIR', '--access-key', 'MYACCESSKEY', '--secret-key', 'MYSECRET', '--region', 'us-east-1', '--debug'] 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_s3 at 0x7fb0916f5220> 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_ddb at 0x7fb091a91170> 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.configure.configure.ConfigureCommand'>> 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x7fb0919f8720> 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x7fb0919fa140> 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function alias_opsworks_cm at 0x7fb091839bc0> 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_history_commands at 0x7fb0917fa6c0> 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.devcommands.CLIDevCommand'>> 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_waiters at 0x7fb091710eb0> 2025-10-15 12:56:59,719 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x7fb0916abb60>> 2025-10-15 12:56:59,719 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/lib/python3.14/site-packages/awscli/data/cli.json 2025-10-15 12:56:59,720 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_types at 0x7fb0917e2140> 2025-10-15 12:56:59,720 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function no_sign_request at 0x7fb0917e24b0> 2025-10-15 12:56:59,720 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_verify_ssl at 0x7fb0917e2400> 2025-10-15 12:56:59,720 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_read_timeout at 0x7fb0917e2610> 2025-10-15 12:56:59,720 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_connect_timeout at 0x7fb0917e2560> 2025-10-15 12:56:59,720 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <built-in method update of dict object at 0x7fb0915ab7c0> 2025-10-15 12:56:59,720 - MainThread - botocore.session - DEBUG - Setting config variable for region to 'us-east-1' 2025-10-15 12:56:59,720 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.27.0 Python/3.14.0 Linux/6.17.1-300.fc43.x86_64 source/x86_64.fedora.43 2025-10-15 12:56:59,720 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['datasync', 'create-location-object-storage', '--server-hostname', 'storage.googleapis.com', '--server-protocol', 'HTTPS', '--server-port', '443', '--bucket-name', 'MYBUCKET', '--subdirectory', 'MYDIR', '--access-key', 'MYACCESSKEY', '--secret-key', 'MYSECRET', '--region', 'us-east-1', '--debug'] 2025-10-15 12:56:59,720 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_timestamp_parser at 0x7fb091710460> 2025-10-15 12:56:59,720 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function register_uri_param_handler at 0x7fb092ae9170> 2025-10-15 12:56:59,720 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_binary_formatter at 0x7fb091be2610> 2025-10-15 12:56:59,721 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function no_pager_handler at 0x7fb091d889e0> 2025-10-15 12:56:59,721 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x7fb091be1d20> 2025-10-15 12:56:59,721 - MainThread - botocore.utils - DEBUG - IMDS ENDPOINT: http://IP.ADD.RE.SS/ 2025-10-15 12:56:59,722 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x7fb0917f9010> 2025-10-15 12:56:59,722 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_json_file_cache at 0x7fb091a881a0> 2025-10-15 12:56:59,725 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/lib/python3.14/site-packages/awscli/botocore/data/datasync/2018-11-09/service-2.json 2025-10-15 12:56:59,730 - MainThread - botocore.hooks - DEBUG - Event building-command-table.datasync: calling handler <function add_waiters at 0x7fb091710eb0> 2025-10-15 12:56:59,733 - MainThread - botocore.hooks - DEBUG - Event building-command-table.datasync: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x7fb0916abb60>> 2025-10-15 12:56:59,733 - MainThread - awscli.clidriver - DEBUG - OrderedDict({'server-hostname': <awscli.arguments.CLIArgument object at 0x7fb0915f6f90>, 'server-port': <awscli.arguments.CLIArgument object at 0x7fb0915eb390>, 'server-protocol': <awscli.arguments.CLIArgument object at 0x7fb0915eb4d0>, 'subdirectory': <awscli.arguments.CLIArgument object at 0x7fb0915ee3f0>, 'bucket-name': <awscli.arguments.CLIArgument object at 0x7fb0915ee520>, 'access-key': <awscli.arguments.CLIArgument object at 0x7fb091765c70>, 'secret-key': <awscli.arguments.CLIArgument object at 0x7fb09173bac0>, 'agent-arns': <awscli.arguments.ListArgument object at 0x7fb0915f70e0>, 'tags': <awscli.arguments.ListArgument object at 0x7fb0915eb610>, 'server-certificate': <awscli.arguments.CLIArgument object at 0x7fb09173bbd0>}) 2025-10-15 12:56:59,733 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.datasync.create-location-object-storage: calling handler <function add_streaming_output_arg at 0x7fb0916f7950> 2025-10-15 12:56:59,733 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.datasync.create-location-object-storage: calling handler <function add_cli_input_json at 0x7fb091be2da0> 2025-10-15 12:56:59,733 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.datasync.create-location-object-storage: calling handler <function add_cli_input_yaml at 0x7fb091be3cc0> 2025-10-15 12:56:59,733 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.datasync.create-location-object-storage: calling handler <function unify_paging_params at 0x7fb091a91850> 2025-10-15 12:56:59,736 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/lib/python3.14/site-packages/awscli/botocore/data/datasync/2018-11-09/paginators-1.json 2025-10-15 12:56:59,736 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.datasync.create-location-object-storage: calling handler <function add_generate_skeleton at 0x7fb0917e07d0> 2025-10-15 12:56:59,736 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.datasync.create-location-object-storage: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputJSONArgument object at 0x7fb0915f7230>> 2025-10-15 12:56:59,736 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.datasync.create-location-object-storage: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputYAMLArgument object at 0x7fb0915f7380>> 2025-10-15 12:56:59,736 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.datasync.create-location-object-storage: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x7fb0915f74d0>> 2025-10-15 12:56:59,736 - MainThread - botocore.hooks - DEBUG - Event building-command-table.datasync_create-location-object-storage: calling handler <function add_waiters at 0x7fb091710eb0> 2025-10-15 12:56:59,736 - MainThread - botocore.hooks - DEBUG - Event building-command-table.datasync_create-location-object-storage: calling handler <bound method AliasSubCommandInjector.on_building_command_table of <awscli.alias.AliasSubCommandInjector object at 0x7fb0916abb60>> 2025-10-15 12:56:59,737 - MainThread - awscli.clidriver - DEBUG - Exception caught in main() Traceback (most recent call last): File "/usr/lib64/python3.14/argparse.py", line 2040, in _parse_known_args2 namespace, args = self._parse_known_args(args, namespace, intermixed) ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.14/argparse.py", line 2337, in _parse_known_args raise ArgumentError(None, _('the following arguments are required: %s') % ', '.join(required_actions)) argparse.ArgumentError: the following arguments are required: --agent-arns

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/lib/python3.14/site-packages/awscli/clidriver.py", line 515, in main return command_table[parsed_args.command](remaining, parsed_args) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.14/site-packages/awscli/clidriver.py", line 652, in call return command_table[parsed_args.operation](remaining, parsed_globals) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib/python3.14/site-packages/awscli/clidriver.py", line 827, in call parsed_args, remaining = operation_parser.parse_known_args(args) ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^ File "/usr/lib/python3.14/site-packages/awscli/argparser.py", line 227, in parse_known_args return super(ArgTableArgParser, self).parse_known_args( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ args, namespace ^^^^^^^^^^^^^^^ ) ^ File "/usr/lib/python3.14/site-packages/awscli/argparser.py", line 95, in parse_known_args parsed, remaining = super(CLIArgParser, self).parse_known_args( ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^ args, namespace ^^^^^^^^^^^^^^^ ) ^ File "/usr/lib64/python3.14/argparse.py", line 2011, in parse_known_args return self._parse_known_args2(args, namespace, intermixed=False) ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/lib64/python3.14/argparse.py", line 2042, in _parse_known_args2 self.error(str(err)) ~~~~~~~~~~^^^^^^^^^^ File "/usr/lib/python3.14/site-packages/awscli/argparser.py", line 130, in error raise ArgParseException(f'{usage_message}\n{error_message}') awscli.argparser.ArgParseException: usage: aws [options] [ ...] [parameters] To see help text, you can run:

aws help aws help aws help

aws: error: the following arguments are required: --agent-arns

ThomasMorrow-SOI avatar Oct 15 '25 17:10 ThomasMorrow-SOI

Thanks for the reply and information. I have reached out to the Datasync Service team in this regard. I will updates as soon as there are updates form the team. Thank you.

Internal Ref: P316196911

adev-code avatar Oct 15 '25 17:10 adev-code

I hit the same issue today and after some time investigating, it turns out by default TasMode defaults to BASIC which does not support agentless. Switching to ENHANCED task mode, fixes the issue.

mvalipour avatar Oct 23 '25 15:10 mvalipour