airflow icon indicating copy to clipboard operation
airflow copied to clipboard
verified publisher icon apache

Status of testing Providers that were prepared on June 15, 2025

Open eladkal opened this issue 7 months ago • 19 comments

Body

I have a kind request for all the contributors to the latest provider distributions release. Could you please help us to test the RC versions of the providers?

The guidelines on how to test providers can be found in

Verify providers by contributors

Let us know in the comments, whether the issue is addressed.

These are providers that require testing as there were some substantial changes introduced:

Provider airbyte: 5.2.0rc1

Provider alibaba: 3.2.0rc1

Provider amazon: 9.9.0rc1

Provider apache.hdfs: 4.10.0rc1

Provider celery: 3.12.0rc1

Provider cncf.kubernetes: 10.6.0rc1

Provider cohere: 1.5.1rc1

Provider common.sql: 1.27.2rc1

Provider databricks: 7.5.0rc1

Provider edge3: 1.1.1rc1

Provider fab: 2.2.1rc1

Provider git: 0.0.3rc1

Provider google: 16.0.0rc1

Provider hashicorp: 4.3.0rc1

Provider microsoft.azure: 12.4.1rc1

Provider mongo: 5.2.0rc1

Provider mysql: 6.3.1rc1

Provider neo4j: 3.9.1rc1

Provider openlineage: 2.4.0rc1

Provider oracle: 4.1.1rc1

Provider pinecone: 2.3.1rc1

Provider sendgrid: 4.1.1rc1

Provider sftp: 5.3.1rc1

Provider slack: 9.1.1rc1

Provider snowflake: 6.4.0rc1

Provider standard: 1.3.0rc1

Provider trino: 6.3.0rc1

Provider weaviate: 3.2.0rc1

All users involved in the PRs: @dominikhei @mobuchowski @codenamelxl @fuatcakici @jsjasonseba @olegkachur-e @eladkal @roach231428 @Crowiant @ellisms @zach-overflow @valentinDruzhinin @willyyang @vuonghoangbntt @romsharon98 @jscheffl

Committer

  • [x] I acknowledge that I am a maintainer/committer of the Apache Airflow project.

eladkal avatar Jun 15 '25 08:06 eladkal

Tested my changes in Alibaba providers. Ran it on some sample DAGs. Everything works well. Thanks https://github.com/apache/airflow/pull/50178 https://github.com/apache/airflow/pull/51513 https://github.com/apache/airflow/pull/51112

jsjasonseba avatar Jun 15 '25 11:06 jsjasonseba

+1 (binding) - Checked SVN, Check in Docker, Reproducible package build, Licenses, Signatures

Tested with Airflow 3.0.2 and 2.11.0 and EdgeExecutor to run some workload all looks good. Bugs seems to be fixed.

Found one problem with breeze in Airflow 3.0.2 with FabAuth manager attempting to add a user in the UI, on save presented me this error. I am not sure whether this is in relation to current release or setup with breeze. (Never tested this before). Creationof user via CLI was working. Image No specific error on API server found except the HTTP 302 on user add in UI.

Also found a small glitch in #51733 but this is not blocking. Will make the error message better for next wave.

Will need to test CNCF K8s Provider more in-depth the next days, from brief tests I see no problems. Due to absense of @AutomationDev85 I'll check his contributions

jscheffl avatar Jun 15 '25 12:06 jscheffl

Hi all,

RE:

Fix remote logging CloudWatch handler initialization and stream name assignment (#51022): @gopidesupavan Linked issues:

Linked Issue #50802: @emiliadecaudin

I've tested it on my end and have confirmed that it is now working. Thank you!

emiliadecaudin avatar Jun 15 '25 15:06 emiliadecaudin

Hi everyone! I've tested on my end: 51098 51495 Everything is working fine.

valentinDruzhinin avatar Jun 15 '25 15:06 valentinDruzhinin

Hi folks, I've tested the following in my local env, and everything looks great. Thank you so much! #50442 #51343 #50755 #51058 #51489 #51265 #50744

nailo2c avatar Jun 15 '25 18:06 nailo2c

Hi all, I've tested the following by running my test DAGs in Docker environment, and can delete, and add batch links for objects in Weaviate Sandbox environment. Thanks! #50735 #50750

sjyangkevin avatar Jun 15 '25 20:06 sjyangkevin

Hi all,

I've tested the HDFS provider using the Airflow 3.0.2 Docker build on my end, and it works well. Thank you!

https://github.com/apache/airflow/pull/50759

roach231428 avatar Jun 16 '25 01:06 roach231428

Installed the RC bits and tested my changes along with few example dags, all good!

CI fixes:

  1. https://github.com/apache/airflow/pull/50968
  2. https://github.com/apache/airflow/pull/49807
  3. https://github.com/apache/airflow/pull/50868
  4. https://github.com/apache/airflow/pull/50869 (after 2.11 release, compat issue)

Actual changes:

  1. https://github.com/apache/airflow/pull/51734

amoghrajesh avatar Jun 16 '25 04:06 amoghrajesh

Installed git provider 0.0.3rc1 and tested work well. #51256

codenamelxl avatar Jun 16 '25 07:06 codenamelxl

Common.messaging has a bug: #51770 -> already fixed in #51774

potiuk avatar Jun 16 '25 09:06 potiuk

Common.messaging has a bug: https://github.com/apache/airflow/issues/51770 -> already fixed in https://github.com/apache/airflow/pull/51774

Common.messaging is not part of this wave.

eladkal avatar Jun 16 '25 11:06 eladkal

🤦

potiuk avatar Jun 16 '25 12:06 potiuk

I have tested all of the adjusted Operators, everything works:

#51236 #51196

dominikhei avatar Jun 16 '25 19:06 dominikhei

Tested my changes and looks good to me. Thanks! #50341 #49458

guan404ming avatar Jun 16 '25 19:06 guan404ming

Tested my changes: https://github.com/apache/airflow/pull/46911 https://github.com/apache/airflow/pull/49704

When testing via breeze with Airflow 3.0.2 and apache-airflow-providers-google==16.0.0rc1 everything seems to work fine, link is also redirecting to the correct logs/filter in the GCP logs explorer:

Image

However, I also installed apache-airflow-providers-google==16.0.0rc1 on our GCP Cloud Composer test instance (Airflow 2.10.5+composer) and I cannot see the extra links (or I don't know where to look in Airflow 2, but I'd expect it to render at the task instance details page under the notes section, I did however test on a pre-existing DAG that uses the operator):

Image Image

ramonvermeulen avatar Jun 17 '25 14:06 ramonvermeulen

Tested my changes and looks good: https://github.com/apache/airflow/pull/51669

kiran2706 avatar Jun 17 '25 17:06 kiran2706

Hi! I just tested mine and ran ok.

#49625

Thanks

josuegen avatar Jun 17 '25 19:06 josuegen

For posterity - following devlist thread. I suggest to remove fab as it contains bug https://github.com/apache/airflow/issues/51854 that was there for 2 releases but we have a chance to fix it in rc2.

potiuk avatar Jun 17 '25 19:06 potiuk

Fix to FAB issue https://github.com/apache/airflow/pull/51858

potiuk avatar Jun 17 '25 20:06 potiuk

All my PRs are fine, thanks !

kacpermuda avatar Jun 18 '25 10:06 kacpermuda

Sample DAGs for #51129 work fine.

ellisms avatar Jun 18 '25 13:06 ellisms

Thank you everyone. Providers are released. Fab provider is excluded from this release and will have rc2

I invite everyone to help improve providers for the next release, a list of open issues can be found here.

eladkal avatar Jun 18 '25 15:06 eladkal

Hi everyone, I know this issue has been closed and resolved, but I noticed that the updated Google provider is still not included in the recently released Airflow 3.0.2 image. While the new provider version works fine, I’ve had to build a custom Docker image to include it. I was hoping it would be bundled with one of the recent default releases 🙂 Thanks.

adeniyistephen avatar Jul 04 '25 15:07 adeniyistephen

Hi everyone, I know this issue has been closed and resolved, but I noticed that the updated Google provider is still not included in the recently released Airflow 3.0.2 image. While the new provider version works fine, I’ve had to build a custom Docker image to include it. I was hoping it would be bundled with one of the recent default releases 🙂 Thanks.

See here https://airflow.apache.org/docs/docker-stack/index.html#fixing-images-at-release-time -> our images are fixed at release time with the dependencies (including providers) that were available at tha time. And yes - if you want to upgrade - it's on you to upgrade and the docs above when you look at details even tells you how you can do it.

Also - we are just now voting on 3.0.3 rc candidate https://lists.apache.org/thread/n1l14lrf8gwcxwnqt81fj8c63ondy0zy and we ask people like you to spend their time and effort to test it and let us know (see the message) if what we release is good. It's a very good use of time of our users if they can help us with that and I heartlly encourage you to do so.

Thanks

potiuk avatar Jul 04 '25 16:07 potiuk

Looking forward to your tests and confirming that 3.0.3 rc3 is what works for you @adeniyistephen -> the issue linked in the message is where you can comment on

potiuk avatar Jul 04 '25 16:07 potiuk

Looking forward to your tests and confirming that 3.0.3 rc3 is what works for you @adeniyistephen -> the issue linked in the message is where you can comment on

Amazing, thanks @potiuk

adeniyistephen avatar Jul 04 '25 19:07 adeniyistephen