SQLAlchemy: Attempt to close session raised an exception at the server: Error during request to server

[freud14-tm]

When using the SQLAlchemy dialect, I get this error when my program closes after running that code:

import os

from sqlalchemy import create_engine


server_hostname = os.getenv("DATABRICKS_SERVER_HOSTNAME")
http_path = os.getenv("DATABRICKS_HTTP_PATH")
access_token = os.getenv("DATABRICKS_TOKEN")
engine = create_engine(
    f"databricks://token:{access_token}@{server_hostname}?http_path={http_path}&catalog=hive_metastore&schema=default",
)
with engine.connect() as connection:
    pass

Attempt to close session raised an exception at the server: Error during request to server

Here are version numbers:

In [1]: import sqlalchemy

In [2]: sqlalchemy.__version__
Out[2]: '1.4.49'

In [3]: from databricks import sql

In [4]: sql.__version__
Out[4]: '2.8.0'

[rpanman-sonatype]

I notice the same when using databricks.sql.client directly via execute and fetchall on the cursor.

The appears to happen as the script exits (tested by inserting a sleep before exit - the error appears after the sleep has timed out).

$ poetry show | grep "databricks\|sql"
databricks-sql-connector 2.8.0      Databricks SQL Connector for Python
sqlalchemy               1.4.49     Database Abstraction Library

[rpanman-sonatype]

I've just tested with connector versions 2.7.0 (the error still occurs) and 2.6.0 (the error does not occur), so looks like something introduced in 2.7.0.

[sebbegg]

Looks like https://github.com/databricks/databricks-sql-python/issues/157 imho