astro deploy --force

[MatrixManAtYrService]

• [x ] I have checked that a similar feature request does not already exist.

✍️ Is your feature request related to a problem? Please describe.

I fixed a bug related to astro-runtime being deployed via helm. Then I was worried:

what if this breaks on astro?

So I made a local build which included my fix, and attempted to test it on astro. This involved invoking astro deploy, which refused to let me push the image because it was based on a local build of astro runtime--not one that was hosted on quay.

🧩 Describe the solution you'd like

I'd like a --force flag on astro deploy which ignores any version or compatibility checks and just deploys the image. Even in cases where it's sure to fail, I still want to be able to do it so that I can look at the wreckage and understand why it was a bad idea in the first place.

🤔 Describe alternatives you've considered

1. I played around with patching an authentic image to match the one that I wanted to test, like so: https://github.com/astronomer/astro-runtime/pull/413#issuecomment-1241286393

...but opened the door for a case if I'm-not-testing-what-I-think-I-am

2. What I wanted to push was an authentic runtime image (or it will be, once my PR is merged). So it seems like this check is just based on the name of the base image, not its contents. Wouldn't it be a more useful check if it actually ran some kind of "should_I_be_deployed" check (in the image) via docker run? That way if I want to force a test image I could just temporarily make that function return True. I can imagine other cases where we'd want to bundle "don't deploy unless..." logic with an image, so having such a function might be handy.

[jesse-amano]

I have another use-case where this would be useful: testing tools and extensions that are meant to be deployed with astro-runtime, such as stellar (the "astro runtime executor"). Astro Runtime's CircleCI build and integrations tests are, naturally, built around being able to try stuff out on the runtime image itself, and bringing in other dependencies (like stellar) at known pinned "versions", and I don't really want that to change. However, for testing bugfixes and new features from stellar that are best (or sometimes only) observed on astro cloud, we'd ideally have a separate build or script that does the opposite: brings in a known, pinned version of astro-runtime, adds the experimental stellar stuff in, and makes it available to the local dev as an image that can be pushed to a deployment.

The current process is much more manual:

1. Create a deployment (either astro-ui or astro-cli) selecting a base image that supports stellar (6.0.0 or higher).
2. In stellar workspace, build a wheel python -m build -w
3. Create an astro workspace (astro dev init if necessary)
4. Move the stellar wheel from the stellar workspace to the separate, astro workspace (just makes it easier to reason about what's going into the image, etc.)
5. Copy in or add to the existing Dockerfile, instructions that COPY in the wheel and install it.
6. Run astro deploy

This isn't especially bad, but it has slightly more steps (and a fair bit more waiting for reconciliation-stuff to happen in between steps) than is convenient for rapid development/testing iteration; and it's difficult to create or use a shareable image (i.e. stellar's CircleCI build can't just push experimental images to a repository that PR reviewers, QA testers, etc. could just pull down and play with; anyone who wants to test a branch of stellar on their own deployment has to follow each of these steps). The solution Matt proposed above doesn't inherently make any of this go faster, but would make certain parts easier to script, reason about, and automate; and removes minor uncertainties (e.g. potential differences between creating a new deployment with this image vs. upgrading from a deployment that had a different image).

[sunkickr]

@MatrixManAtYrService @jesse-amano this seems like more of feature that should be available to internal members of astronomer than one that is available to customers. If we created a way for astronomer org memebers to do that is would that work?