Asking for best practice on handling multiple scheduled job in AWS CoPilot

[rickychew77]

Currently I have multiple scheduled jobs to be run with just different command and schedule, and the documentation i have read here, current way is having directory structure below to organize it, which can be a long list and repetitive.

- copilot
  - project-app
    - manifest.yml 
  - project-scheduled-task-1
    - manifest.yml 
  - project-scheduled-task-2
    - manifest.yml
  - project-scheduled-task-3
    - manifest.yml
  - project-scheduled-task-4
    - manifest.yml

Is there a way to just control these scheduled task with just multiple manifest.yml within same directory or better yet using only one manifest.yml?

[rickychew77]

Like for example, copilot job init command here can it have additional flag so it tells copilot which .yml files to pick up instead of just looking for default manifest.yml file? it's generally for easier maintenance!

[Lou1415926]

@rickychew77 I think what you are looking for is similar to https://github.com/aws/copilot-cli/issues/2699 (with which, you would be able to maintain just manifest.yml and parameterize the command and schedule) and https://github.com/aws/copilot-cli/issues/2388 (with which you wouldn't be confined to the copilot/job-1+ directory structure).

These are not supported today yet unfortunately 😞 ! You could do this with means outside of Copilot, such as having only the differentiating fields in the individual manifests ⬇️

on:
  schedule: @daily 
command: "some command"

and append the shared job config to the files automatically as a part of the CI/CD...or store the other configs as a JSON string shell env var and refer to the shell env var from the manifest... Admittedly not super great options 😂

I think the --manifest flag would probably help a great ton in your case 👍🏼 !