databricks-sql-python icon indicating copy to clipboard operation
databricks-sql-python copied to clipboard
verified publisher icon databricks

Enhance Arrow to Pandas conversion with type overrides and additional kwargs

Open madhav-db opened this issue 7 months ago • 7 comments

  • Introduced _arrow_pandas_type_override and _arrow_to_pandas_kwargs in Connection class for customizable dtype mapping and DataFrame construction parameters.
  • Updated ResultSet to utilize these new options during conversion from Arrow tables to Pandas DataFrames.
  • Added unit tests to validate the new functionality, including scenarios for type overrides and additional kwargs handling.

What type of PR is this?

  • [ ] Refactor
  • [x] Feature
  • [ ] Bug Fix
  • [ ] Other

Description

How is this tested?

  • [x] Unit tests
  • [ ] E2E Tests
  • [ ] Manually
  • [ ] N/A

Related Tickets & Documents

#578

madhav-db avatar May 30 '25 06:05 madhav-db