chart icon indicating copy to clipboard operation
chart copied to clipboard

S3 upload

Open timetinytim opened this issue 1 year ago • 1 comments

Added a job to upload assets to an S3 bucket on helm chart creation. Tested this in an isolated situation outside of a chart, and the job successfully uploads to exoscale, so the actual job definition works. Haven't tested it in the actual chart itself yet.

One note: this uploads the entire /public folder. Not sure if this is the "correct" thing to do or not, or if just /public/assets should be uploaded. Let me know if this should be changed.

timetinytim avatar Sep 23 '24 08:09 timetinytim

After some tweaks and testing, I updated how the transfer is done. Setting --transfers to something higher than the default of 4 does significantly increases speed, and --fast-list helps speed things up a little bit, though only marginally in this case. With the current configuration, and the moment:

Transferring the whole /public folder (just for comparison's sake) Initial copy: ~30s Re-run (i.e. checking existing files): ~60s

Transferring just /public/assets and /public/packs Initial copy: ~17s Re-run (i.e. checking existing files): ~22s

@renchap Let me know if this works. I think this is much more reasonable than it was before, and good enough to merge.

timetinytim avatar Sep 23 '24 12:09 timetinytim