databricks-sql-python icon indicating copy to clipboard operation
databricks-sql-python copied to clipboard

Allow passing parameters into `to_pandas` call

Open j-bennet opened this issue 7 months ago • 1 comments

When creating a Connection, currently it's possible to configure some aspects of pandas / arrow, such as _use_arrow_native_complex_types and _use_arrow_native_timestamps, but _convert_arrow_table uses a hardcoded type mapper that's impossible to change:

https://github.com/databricks/databricks-sql-python/blob/0947b9aa194d37198fa0b05147f50e4682a5643f/src/databricks/sql/client.py#L1349-L1362

Similarly, there's no way to pass other parameters to the to_pandas call here:

https://github.com/databricks/databricks-sql-python/blob/0947b9aa194d37198fa0b05147f50e4682a5643f/src/databricks/sql/client.py#L1366-L1370

Please consider extending the connection parameters to add more flexibility.

j-bennet avatar May 29 '25 23:05 j-bennet