amazon-genomics-cli icon indicating copy to clipboard operation
amazon-genomics-cli copied to clipboard

only specific instance types and not instance families can be listed in `context.instanceTypes`

Open wleepang opened this issue 3 years ago • 2 comments

Describe the Bug

The documentation for context configuration implies that one can supply instance family names (e.g. "c5", "m5", "r5", etc) in the list of instanceTypes used by a context.

See: https://github.com/aws/amazon-genomics-cli/blob/6ed94111ea167d9177e315aafabc1e1b9752452c/site/content/en/docs/Concepts/contexts.md?plain=1#L81-L90

However, if I try to deploy a context with the following config:

contexts:
    cpuctxminiwdl:
        engines:
            - type: wdl
              engine: miniwdl
        instanceTypes:
          - c5
          - m5
          - r5

the context fails to deploy and I get the following error:

2022-04-29T22:34:03Z ✘   error="1 context deployment failures"
Error: an error occurred invoking 'context deploy'
with variables: {contexts:[] deployAll:false}
caused by: 1 context deployment failures
suggestion: To resolve failure 1, determine the cause of: exit status 1

Steps to Reproduce

Create a new agc project.

Update the agc-project.yaml file to contain:

name: test
schemaVersion: 1
contexts:
    cpuctxminiwdl:
        engines:
            - type: wdl
              engine: miniwdl
        instanceTypes:
          - c5
          - m5
          - r5

Run:

agc context deploy -v cpuctxminiwdl

Relevant Logs

2022-04-29T22:33:53Z ↓
2022-04-29T22:34:03Z 𝒊  /home/ec2-user/.agc/cdk/node_modules/aws-cdk-lib/aws-ec2/lib/instance-types.ts:666
2022-04-29T22:34:03Z 𝒊        throw new Error('Malformed instance type identifier');
2022-04-29T22:34:03Z 𝒊              ^
2022-04-29T22:34:03Z 𝒊  Error: Malformed instance type identifier
2022-04-29T22:34:03Z 𝒊      at InstanceType.get architecture [as architecture] (/home/ec2-user/.agc/cdk/node_modules/aws-cdk-lib/aws-ec2/lib/instance-types.ts:666:13)
2022-04-29T22:34:03Z 𝒊      at /home/ec2-user/.agc/cdk/lib/util/instance-types.ts:179:83
2022-04-29T22:34:03Z 𝒊      at Array.filter (<anonymous>)
2022-04-29T22:34:03Z 𝒊      at Object.exports.getInstanceTypesForBatch (/home/ec2-user/.agc/cdk/lib/util/instance-types.ts:179:45)
2022-04-29T22:34:03Z 𝒊      at Batch.renderComputeEnvironment (/home/ec2-user/.agc/cdk/lib/constructs/batch.ts:203:24)
2022-04-29T22:34:03Z 𝒊      at new Batch (/home/ec2-user/.agc/cdk/lib/constructs/batch.ts:93:36)
2022-04-29T22:34:03Z 𝒊      at MiniwdlEngineConstruct.renderBatch (/home/ec2-user/.agc/cdk/lib/stacks/engines/miniwdl-engine-construct.ts:117:12)
2022-04-29T22:34:03Z 𝒊      at new MiniwdlEngineConstruct (/home/ec2-user/.agc/cdk/lib/stacks/engines/miniwdl-engine-construct.ts:36:30)
2022-04-29T22:34:03Z 𝒊      at ContextStack.renderMiniwdlStack (/home/ec2-user/.agc/cdk/lib/stacks/context-stack.ts:87:5)
2022-04-29T22:34:03Z 𝒊      at new ContextStack (/home/ec2-user/.agc/cdk/lib/stacks/context-stack.ts:37:14)
2022-04-29T22:34:03Z 𝒊  Subprocess exited with error 1
2022-04-29T22:34:03Z ✘  Failed to deploy context 'cpuctxminiwdl'. Below is the log for that deployment
2022-04-29T22:34:03Z ✘  /home/ec2-user/.agc/cdk/node_modules/aws-cdk-lib/aws-ec2/lib/instance-types.ts:666
2022-04-29T22:34:03Z ✘        throw new Error('Malformed instance type identifier');
2022-04-29T22:34:03Z ✘              ^
2022-04-29T22:34:03Z ✘  Error: Malformed instance type identifier
2022-04-29T22:34:03Z ✘      at InstanceType.get architecture [as architecture] (/home/ec2-user/.agc/cdk/node_modules/aws-cdk-lib/aws-ec2/lib/instance-types.ts:666:13)
2022-04-29T22:34:03Z ✘      at /home/ec2-user/.agc/cdk/lib/util/instance-types.ts:179:83
2022-04-29T22:34:03Z ✘      at Array.filter (<anonymous>)
2022-04-29T22:34:03Z ✘      at Object.exports.getInstanceTypesForBatch (/home/ec2-user/.agc/cdk/lib/util/instance-types.ts:179:45)
2022-04-29T22:34:03Z ✘      at Batch.renderComputeEnvironment (/home/ec2-user/.agc/cdk/lib/constructs/batch.ts:203:24)
2022-04-29T22:34:03Z ✘      at new Batch (/home/ec2-user/.agc/cdk/lib/constructs/batch.ts:93:36)
2022-04-29T22:34:03Z ✘      at MiniwdlEngineConstruct.renderBatch (/home/ec2-user/.agc/cdk/lib/stacks/engines/miniwdl-engine-construct.ts:117:12)
2022-04-29T22:34:03Z ✘      at new MiniwdlEngineConstruct (/home/ec2-user/.agc/cdk/lib/stacks/engines/miniwdl-engine-construct.ts:36:30)
2022-04-29T22:34:03Z ✘      at ContextStack.renderMiniwdlStack (/home/ec2-user/.agc/cdk/lib/stacks/context-stack.ts:87:5)
2022-04-29T22:34:03Z ✘      at new ContextStack (/home/ec2-user/.agc/cdk/lib/stacks/context-stack.ts:37:14)
2022-04-29T22:34:03Z ✘  Subprocess exited with error 1



2022-04-29T22:34:03Z ✘   error="1 context deployment failures"
Error: an error occurred invoking 'context deploy'
with variables: {contexts:[] deployAll:false}
caused by: 1 context deployment failures
suggestion: To resolve failure 1, determine the cause of: exit status 1

Expected Behavior

I can use instance familiy names (e.g. "c5", "m5", "r5", etc) in addition to explicitly listing specific instance types, as the documentation states.

Actual Behavior

Using instance family names is not accepted. Deploying a context throws and error. Reverting to specific instance types e.g "c5.xlarge" works.

Screenshots

Additional Context

Operating System: Amazon Linux 2 AGC Version: 1.3.0 Was AGC setup with a custom bucket: no Was AGC setup with a custom VPC: no

wleepang avatar Apr 29 '22 22:04 wleepang

Greetings! Sorry to say but this is a very old issue that is probably not getting as much attention as it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to open a new one.

github-actions[bot] avatar Jul 29 '22 00:07 github-actions[bot]

Upvoted. Please at least update in the documentation for now.

16967143 avatar Jul 31 '23 08:07 16967143