databricks-sql-python Strict numpy dependency

Hi!

I'm running into an issue that this package requires a very specific numpy package 1.23.4. Aside from wondering why it needs numpy in the first place (I didn't dive into that), I'm wondering if this needs to be this specific.

Context: We are using NixPkgs to provide reproducible environments. I'm trying to compile the latest dbt-databricks version and this is a dependency. From the latest stable NixPkgs I get numpy version 1.23.3. I can override it but building this specific version of numpy myself is quite heavy. And it makes me wonder if it can be solved in a different way by relaxing the constraint :-)

Thanks for thinking along!

Feb 10 '23 09:02 bjgbeelen

Same here, I'm having problems updating the Databricks driver in Apache Superset because of this. Can we relax the versioning here?

https://github.com/databricks/databricks-sql-python/blob/e3f3a6febc903399ceb2db858faa142c754ad38c/pyproject.toml#L22-L25

Feb 17 '23 21:02 betodealmeida

There's a PR for this here: #74 we're working on merging it ASAP.

Feb 22 '23 16:02 susodapop

Also @betodealmeida we'd like to improve the Superset experience which currently doesn't use our built-in SQLAlchemy dialect (just released in v2.4.0). Are you interested in helping us test the new version?

Feb 22 '23 16:02 susodapop

Also @betodealmeida we'd like to improve the Superset experience which currently doesn't use our built-in SQLAlchemy dialect (just released in v2.4.0). Are you interested in helping us test the new version?

Yeah, I'm happy to bump the version and test it! Will do it this weekend.

Feb 24 '23 18:02 betodealmeida

@betodealmeida Unfortunately that won't work! Superset needs to be configured to use our in-built dialect. At present if you just bump the version of databricks-sql-connector nothing will change. Once we have concrete steps for testing I'll ping you here.

Feb 24 '23 19:02 susodapop