upload-cloud-storage icon indicating copy to clipboard operation
upload-cloud-storage copied to clipboard

Want to replace all files in a folder rather than merge.

Open Michael-S-Hammond opened this issue 4 years ago • 5 comments

TL;DR

When I upload a new version of my website, I want to remove all existing files and replace with my new ones. The current design merges my new files into the existing contents which leaves unneeded (and likely deprecated) files behind.

Design

Action YAML

- name: Upload to Google
  uses: google-github-actions/upload-cloud-storage@main
  with:
    credentials: ${{ secrets.credentials}}
    path: 'sourcedir'
    destination: ${{ secrets.bucket}}
    parent: false
    remove: true

In the above, I would expect the contents of the entire bucket to be deleted. If parent is true, then I would expect the contents of the appropriate folder to be deleted.

Resources https://github.com/google-github-actions/upload-cloud-storage/issues/40

  • Link to answer of current implementation

gsutil rm gs://bucket/**

  • From https://cloud.google.com/storage/docs/gsutil/commands/rm, one example of how to remove contents from a bucket.

Michael-S-Hammond avatar Jul 13 '21 17:07 Michael-S-Hammond

Any updates on this?

danburonline avatar Sep 19 '21 16:09 danburonline

I would love to see this!

However, you can use a workaround of running gsutil rsync instead of using this action. You will need to authorize as you currently do (a service account is required as of writing this - there's a warning in that readme). Then you setup gcloud and run the gsutil rsync.

Here is an example. It assumes that you are uploading the ./out directory to a bucket:

    - id: 'auth'
      uses: 'google-github-actions/auth@v0'
      with:
        credentials_json: '${{ secrets.A_SECRET_SERVICE_ACCOUNT_JSON }}'
    - name: 'Set up Cloud SDK'
      uses: 'google-github-actions/setup-gcloud@v0'

    - name: 'Upload Site'
      run: 'gsutil rsync -d -r ./out gs://bucket.example.com'

Note that this is a bit dangerous - if your directory is empty you will quickly delete all your files. You might add a step like this as a safeguard before the upload:

    - name: 'Validate directory not empty'
      run: '[ "$(ls -A ./out)" ] && echo "Files exist - all good." || (echo "Directory is empty - aborting." && exit 1)'

# Put the actions in the first example here.

markwatson avatar Jan 17 '22 06:01 markwatson

any updates on this?

AlexFragotsis avatar Jun 02 '22 20:06 AlexFragotsis

Just for that reason I created my own action: https://github.com/patrickwyler/gcs-bucket-sync-action

patrickwyler avatar Jun 16 '22 06:06 patrickwyler

any update on this?

Jplus2 avatar Jun 30 '22 02:06 Jplus2

Hey folks - like I said in https://github.com/google-github-actions/upload-cloud-storage/issues/4#issuecomment-1356501676, this GitHub Action is optimized for one way syncing from GitHub Actions to Google Cloud Storage. For syncing, we recommend using google-github-actions/setup-gcloud and using the gcloud storage or gsutil components to rsync. This gives you full control over which target is the source of truth, deletion criteria, and more.

sethvargo avatar Dec 17 '22 23:12 sethvargo