databricks-sql-python
databricks-sql-python copied to clipboard
Allow passing parameters into `to_pandas` call
When creating a Connection, currently it's possible to configure some aspects of pandas / arrow, such as _use_arrow_native_complex_types and _use_arrow_native_timestamps, but _convert_arrow_table uses a hardcoded type mapper that's impossible to change:
https://github.com/databricks/databricks-sql-python/blob/0947b9aa194d37198fa0b05147f50e4682a5643f/src/databricks/sql/client.py#L1349-L1362
Similarly, there's no way to pass other parameters to the to_pandas call here:
https://github.com/databricks/databricks-sql-python/blob/0947b9aa194d37198fa0b05147f50e4682a5643f/src/databricks/sql/client.py#L1366-L1370
Please consider extending the connection parameters to add more flexibility.