Allow multiple regions to be specified on invocation

[ablackrw]

Describe the feature

For selected subcommands (such as ses list-identities or ec2 describe-vpcs), allow the user to specify multiple regions for execution using the --region command line switch. Rather than specifying a single region name, the user would provide a comma separated list of region names. For this enhancement to be relevant, the subcommand would need to

1. Be region specific
2. Allow execution without a specifying an instance identifier

When multiple regions are specified, the output would be generated in map form, with the map keys being the region name where the command was executed and the map values being the output of the command execution.

Use Case

From time to time, we need to gather information about our AWS environment across all of our accounts and regions. To gather this information, we resort to using a shell looping construct similar to the following

echo -n "{" > ses.json
for R in us-east-2 us-east-1 us-west-1 us-west-2 af-south-1 ap-south-1 ap-northeast-3 ap-northeast-2 ap-southeast-1 ap-southeast-2 ap-northeast-1 ca-central-1 eu-central-1 eu-west-1 eu-west-2 eu-south-1 eu-west-3 eu-north-1 me-south-1 sa-east-1
do 
  echo -n "${R}:" | tee -a ses.json
  aws --output=json --region=${R} ses list-identities >> ses.json
  echo "," >> ses.json
done
echo "}" >> ses.json

A problem with the above construct is that it fails to produce syntactically correct json (though the output is close)

The proposed feature would allow us to make the invocation as follows aws --output=json --region=us-east-2,us-east-1,us-west-1,us-west-2,af-south-1,ap-south-1,ap-northeast-3,ap-northeast-2,ap-southeast-1,ap-southeast-2,ap-northeast-1,ca-central-1,eu-central-1,eu-west-1,eu-west-2,eu-south-1,eu-west-3,eu-north-1,me-south-1,sa-east-1 ses list-identities > ses.json This would have the side benefit of producing syntactically correct json.

Proposed Solution

A naive (and untested) implementation of the change is as follows

index 304cebbe6..1baef958e 100644
--- a/awscli/clidriver.py
+++ b/awscli/clidriver.py
@@ -645,12 +645,23 @@ class CLIOperationCaller(object):
             value is returned.
 
         """
-        client = self._session.create_client(
-            service_name, region_name=parsed_globals.region,
-            endpoint_url=parsed_globals.endpoint_url,
-            verify=parsed_globals.verify_ssl)
-        response = self._make_client_call(
-            client, operation_name, parameters, parsed_globals)
+        regions = parsed_globals.region.split(',')
+        response = {}
+        if 1 == regions.count:
+            client = self._session.create_client(
+                service_name, region_name=parsed_globals.region,
+                endpoint_url=parsed_globals.endpoint_url,
+                verify=parsed_globals.verify_ssl)
+            response = self._make_client_call(
+                client, operation_name, parameters, parsed_globals)
+        else:
+            for region in regions:
+                client = self._session.create_client(
+                    service_name, region_name=region,
+                    endpoint_url=parsed_globals.endpoint_url,
+                    verify=parsed_globals.verify_ssl)
+                response[region] = self._make_client_call(
+                    client, operation_name, parameters, parsed_globals)
         self._display_response(operation_name, response, parsed_globals)
         return 0

Other Information

No response

Acknowledgements

• [X] I may be able to implement this feature request
• [ ] This feature might incur a breaking change

CLI version used

aws-cli/2.7.7 Python/3.9.13 Darwin/21.6.0 source/arm64 prompt/off

Environment details (OS name and version, etc.)

Darwin ablack0321 21.6.0 Darwin Kernel Version 21.6.0: Sat Jun 18 17:05:47 PDT 2022; root:xnu-8020.140.41~1/RELEASE_ARM64_T8101 arm64

[tim-finnigan]

Hi @ablackrw thanks for the feature request. I don’t think this is likely to be considered because it could introduce a lot of complexity to benefit just a few commands/use cases. Have you tried using something like xargs to chain multiple CLI commands together? Also using an AWS SDK like boto3 might be easier to accomplish what you’re doing with multiple regions rather than using a bash script.

[ablackrw]

@tim-finnigan - I'm not certain xargs would be of much assistance, as it wouldn't help with the structuring of the output of multiple invocations of the aws cli into something that is syntactically correct json. While it might not strictly matter that the result is syntactically correct, the ability to identify which region the command was executed in is a must.

When I ran the bash script from the use case, I wasn't just iterating over the dozen or so regions listed. The loop was actually executed within a larger loop iterating over ~50 AWS accounts. Without the ability to record the output in a structured manner, it becomes difficult to identify the account and region associated with a given piece of output. In addition, there is a time cost associated with running such an ad-hoc loop. While the overhead of a single invocation of the AWS cli is minimal, it quickly compounds when you are invoking the API 500+ times in a row.

The invocation of the ses list-identitites sub-command in the example was just the most recent time I've had to perform such an iteration. Other times I've had to perform this sort of iteration, the sub-command has been ec2 describe-vpcs or ec2 describe-instance. While it would be possible to go down the SDK path, this could quickly lead to the reinvention of the wheel if a CLI parser were added.