PyHive Incompatibility with SQLAlchemy 2.0.0+

SQLAlchemy 2.0.0 has been released on the 26th of January 2023, and it's incompatible with PyHive for connections to Presto, Trino and Hive:

from sqlalchemy import create_engine
create_engine('presto://foo:bar@host')

# results in

  File "/some/path/pyhive/sqlalchemy_presto.py", line 16, in <module>
    from sqlalchemy.databases import mysql
ModuleNotFoundError: No module named 'sqlalchemy.databases'

Similarly,

from sqlalchemy import create_engine
create_engine('trino://foo:bar@host')

# results in

  File "/some/path/sqlalchemy/util/langhelpers.py", line 366, in load
    "Can't load plugin: %s:%s" % (self.group, name)
sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:trino

and

from sqlalchemy import create_engine
create_engine('hive://foo:bar@host')

# results in
  File "/some/path/pyhive/sqlalchemy_hive.py", line 16, in <module>
    from sqlalchemy import processors
ImportError: cannot import name 'processors' from 'sqlalchemy' (/some/path/sqlalchemy/__init__.py)

Pinning SQLAlchemy<2.0.0 resolves the issue.

Feb 03 '23 12:02 PApostol

I've added a pull request to fix this:

https://github.com/dropbox/PyHive/pull/447

I'm not able to sign the CLA at the moment, so the pull won't get approved - but the changes are almost trivial - if you inspect the request you can see how it's done.

Mar 06 '23 12:03 james1465

I've added a pull request to fix this:

#447

I'm not able to sign the CLA at the moment, so the pull won't get approved - but the changes are almost trivial - if you inspect the request you can see how it's done.

Nice work! Although I suspect for complete compatibility there needs to be a fix in pyhive/sqlalchemy_trino.py and pyhive/sqlalchemy_presto.py as well.

Mar 06 '23 14:03 PApostol

Could it be merged please ?

Jun 09 '23 08:06 belgacea

This PR https://github.com/dropbox/PyHive/pull/448 includes the change and has been merged. But then got reverted here: https://github.com/dropbox/PyHive/commit/df03bef66500541fa921ec3614ec06a15ca17615

@bkyryliuk what's the situation atm?

Jun 20 '23 09:06 glungley

I am working on a comprehensive PR to add SQLAlchemy 2.0 Support. Should be ready soon.

Jun 20 '23 10:06 mdeshmu

PR raised https://github.com/dropbox/PyHive/pull/457

Jun 27 '23 14:06 mdeshmu

PR https://github.com/dropbox/PyHive/pull/457 is merged.

Jul 12 '23 14:07 mdeshmu

@PApostol @james1465 @belgacea @glungley

PyHive is now compatible with SQLAlchemy 2.0. See https://github.com/dropbox/PyHive/pull/457 This is released in 0.7.1.dev0. You are requested to test with the dev version and report any bugs in the PyHive GitHub repository before 0.7.1 is released in a month or so.

Jul 14 '23 15:07 mdeshmu

I have tried PyHive 0.7.1.dev0 with SQLAlchemy 2.0.19, it appears to work fine!

Jul 18 '23 14:07 PApostol

PyHive 0.7.0 is released which includes Python 3.11 and SQLAlchemy 2.0 support.

Aug 19 '23 02:08 mdeshmu