clusterfuzz icon indicating copy to clipboard operation
clusterfuzz copied to clipboard

`gcloud.app.deploy` no longer accepts `python37`.

Open nguyendon opened this issue 1 year ago • 13 comments

Python 3.7 is end of support and Gcloud is no longer accepting it as a runtime. While running the butler.py script as prescribed in the production setup documentation, I am running into an error

| ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: Error(s) encountered validating runtime. Runtime python37 is end of support and no longer allowed. Please use the latest Python runtime for App Engine Standard..
| Return code is non-zero (1).
gcloud deployment failed, retrying...
Running: gcloud app deploy --no-stop-previous-version --quiet --project=foobar  src/appengine/index.yaml src/appengine/app.yaml src/appengine/cron.yaml src/appengine/cron-service.yaml
| Services to deploy:
|
| descriptor:                  [/clusterfuzz/src/appengine/app.yaml]
| source:                      [/clusterfuzz/src/appengine]
| target project:              [foobar]
| target service:              [default]
| target version:              [20240227t222323]
| target url:                  [https://foobaruc.r.appspot.com]
| target service account:      [[email protected]]
|
|
| descriptor:                  [/clusterfuzz/src/appengine/cron-service.yaml]
| source:                      [/clusterfuzz/src/appengine]
| target project:              [foobar]
| target service:              [cron-service]
| target version:              [20240227t222323]
| target url:                  [https://cron-service-dot-foobar.uc.r.appspot.com]
| target service account:      [[email protected]]
|
|
| Configurations to update:
|
| descriptor:      [/clusterfuzz/src/appengine/index.yaml]
| type:            [datastore indexes]
| target project:  [foobar]
|
|
| descriptor:      [/clusterfuzz/src/appengine/cron.yaml]
| type:            [cron jobs]
| target project:  [foobar]
|
|
| Beginning deployment of service [default]...
| ╔════════════════════════════════════════════════════════════╗
| ╠═ Uploading 0 files to Google Cloud Storage                ═╣
| ╚════════════════════════════════════════════════════════════╝
| File upload done.
| ERROR: (gcloud.app.deploy) INVALID_ARGUMENT: Error(s) encountered validating runtime. Runtime python37 is end of support and no longer allowed. Please use the latest Python runtime for App Engine Standard..
| Return code is non-zero (1).
Failed to deploy after 3 retries.

The error is throw by this line:

gcloud app deploy --no-stop-previous-version --quiet --project=foobar  src/appengine/index.yaml src/appengine/app.yaml src/appengine/cron.yaml src/appengine/cron-service.yaml

Where it looks like there are some autogenerated files src/appengine/app.yaml and src/appengine/cron-service.yaml that are requesting runtime: python37.

nguyendon avatar Feb 28 '24 02:02 nguyendon

Yes sorry for this, we're currently using an exception to get around this issue. Is there a process for non-Google users to get this exception? We'd like to upgrade to 3.11 but I can't say it's so easy for us to do so.

jonathanmetzman avatar Feb 29 '24 22:02 jonathanmetzman

ACtually, you can really help us move to 3.11 if youre running ClusterFuzz on Windows. Are you?

jonathanmetzman avatar Feb 29 '24 22:02 jonathanmetzman

Is there a process for non-Google users to get this exception?

Was this a question for me or someone else?

ACtually, you can really help us move to 3.11 if youre running ClusterFuzz on Windows. Are you?

I am not running Windows, though could try to get something up on a Windows machine. Is support on Windows the last piece or is there another reason Windows is needed?

nguyendon avatar Mar 01 '24 02:03 nguyendon

It was the last piece when I last attempted this (August). I don't yet have a timeline for when I will do the upgrade sadly, but I'll discuss this internally.

jonathanmetzman avatar Mar 01 '24 02:03 jonathanmetzman

Was this a question for me or someone else?

For you. I don't know if you can get an exception, but the email cloud sent me brought me to a Google-only process. Did you get a similar email about the deprecation of 3.7?

jonathanmetzman avatar Mar 01 '24 02:03 jonathanmetzman

It was the last piece when I last attempted this (August). I don't yet have a timeline for when I will do the upgrade sadly, but I'll discuss this internally.

Do you have a branch with your attempted changes? I can look when I get some cycles - hoping there isn't too much needed in addition for Windows. I think even partial support for 3.11 would be valuable because 3.7 is hard-blocked right now.

For you. I don't know if you can get an exception, but the email cloud sent me brought me to a Google-only process. Did you get a similar email about the deprecation of 3.7?

I did not get an email though it doesn't mean it didn't go out. We are picking up our CF instance that was deployed a few years ago by a different implementer so I wouldn't have been associated in any system with ClusterFuzz or GCP. Is there any way you could check with the cloud team if there is a public process?

nguyendon avatar Mar 01 '24 02:03 nguyendon

https://github.com/google/clusterfuzz/issues/3809 is related to this, although it's possible we can support some kind of appengine deployment on new Pythons prior to that because the requirement of bundling all dependencies does not exist there.

oliverchang avatar Mar 06 '24 06:03 oliverchang

@jonathanmetzman would you have that branch handy where you started the upgrade?

nguyendon avatar Mar 13 '24 15:03 nguyendon

#3809 is related to this, although it's possible we can support some kind of appengine deployment on new Pythons prior to that because the requirement of bundling all dependencies does not exist there.

I can confirm that appengine deployment on python3.8 seems to work for release v2.6.0 Wasn't able to get master working due to issues with grpc library

bkosciarz avatar Mar 19 '24 21:03 bkosciarz

@jonathanmetzman would you have that branch handy where you started the upgrade?

It was a few PRs, I'd say the state before this PR reverted everything was the most complete, it worked everywhere (as far as I could tell, it's possible that Mac wasn't working at all for me to notice breakages at the time), but windows.

jonathanmetzman avatar Mar 19 '24 21:03 jonathanmetzman

Hey team, we're facing the same issue here as well and support for python >3.7 would be greatly appreciated.

quanyang avatar May 29 '24 09:05 quanyang

should we use 3.10 version of python ??

gittymadman avatar Jul 24 '24 07:07 gittymadman

We're in the process of getting 3.11 working.

jonathanmetzman avatar Jul 24 '24 16:07 jonathanmetzman