databricks-sql-python icon indicating copy to clipboard operation
databricks-sql-python copied to clipboard

`Cursor().description` reports NULL (VOID) and INTERVAL fields as `'string'`

Open alexmalins opened this issue 1 year ago • 1 comments

When querying NULL or INTERVAL columns from a Databricks SQL warehouse, the connector Cursor().description property reports the type code of the columns as a 'string'. If I understand things correctly, I'd have expected it to return 'void' and 'interval', reflecting the underlying data type in Databricks SQL.

There is no definition for VOID and INTERVAL types here.

Minimal code example:

from databricks import sql

with sql.connect(
    server_hostname = os.getenv("DATABRICKS_HOST"),
    http_path = os.getenv("DATABRICKS_HTTP_PATH"),
    access_token = os.getenv("DATABRICKS_TOKEN"),
) as connection:

    with connection.cursor() as cursor:
        cursor.execute("SELECT NULL, CAST(NULL AS VOID), INTERVAL '13' MONTH")
        print(cursor.description)
        print(cursor.fetchall())
        print(cursor.description)
[('NULL', 'string', None, None, None, None, None), ('CAST(NULL AS VOID)', 'string', None, None, None, None, None), ("INTERVAL '13' MONTH", 'string', None, None, None, None, None)]
[Row(NULL=None, CAST(NULL AS VOID)=None, INTERVAL '13' MONTH='1-1')]
[('NULL', 'string', None, None, None, None, None), ('CAST(NULL AS VOID)', 'string', None, None, None, None, None), ("INTERVAL '13' MONTH", 'string', None, None, None, None, None)]

alexmalins avatar Jan 28 '24 07:01 alexmalins

Thanks for the report. The code you linked to is actually part of the sqlalchemy dialect so it doesn't affect anything related to the cursor. I'll have a fix for this oddity in the next couple of weeks.

susodapop avatar Jan 29 '24 18:01 susodapop