policyengine-us icon indicating copy to clipboard operation
policyengine-us copied to clipboard

Add self-hosted GCP runner for CI/CD workflows

Open baogorek opened this issue 2 months ago • 2 comments

Summary

  • Migrates all GitHub Actions workflows from ubuntu-latest to self-hosted GCP runner
  • Improves performance and resource availability for CI/CD jobs
  • Uses n2-standard-16 spot instance (64GB RAM, 32GB swap) in policyengine-research GCP project

Changes

  • Updated .github/workflows/pr.yaml - all 5 jobs now use runs-on: self-hosted
    • Lint
    • check-version
    • Quick-Feedback
    • YAML-NonStructural
    • Structural-and-Python
  • Updated .github/workflows/push.yaml - all 6 jobs now use runs-on: self-hosted
    • Lint
    • versioning
    • YAML-NonStructural
    • Structural-and-Python
    • Publish
    • Deploy
  • Added changelog entry documenting infrastructure change

Test plan

  • [ ] Verify self-hosted runner is online and registered in GitHub Actions
  • [ ] Push this PR and confirm workflows pick up jobs on self-hosted runner
  • [ ] Monitor all CI jobs complete successfully on self-hosted environment
  • [ ] Verify GCP service account authentication still works
  • [ ] Confirm PyPI publish and deployment steps work correctly

Infrastructure notes

Runner VM is managed via:

  • VM name: pe-us-runner
  • Start/stop: gcloud compute instances start/stop pe-us-runner --zone=us-central1-a
  • Cost: ~$180/month running, ~$8/month stopped
  • Setup documented in ~/devl/code-snippets/gcp/github-runners/

🤖 Generated with Claude Code

baogorek avatar Nov 12 '25 20:11 baogorek

Do the github check now run sequentially? I believe in the current version they all run simultaneously

PavelMakarchuk avatar Nov 12 '25 20:11 PavelMakarchuk

Do the github check now run sequentially? I believe in the current version they all run simultaneously

If so, this is adding about an hour to the entire runtime of the tests

PavelMakarchuk avatar Nov 12 '25 21:11 PavelMakarchuk

Do the github check now run sequentially? I believe in the current version they all run simultaneously

@baogorek lmk if you have thoughts on runtime, maybe I misread the logs when they were running

PavelMakarchuk avatar Nov 23 '25 09:11 PavelMakarchuk