chdb icon indicating copy to clipboard operation
chdb copied to clipboard

Support executing queries without pyarrow when it's missing

Open wudidapaopao opened this issue 7 months ago • 2 comments

Fix https://github.com/chdb-io/chdb/issues/386

Changelog category (leave one):

  • New Feature

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

...

Documentation entry for user-facing changes

  • [ ] Documentation is written (mandatory for new features)

Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/

CI Settings

NOTE: If your merge the PR with modified CI you MUST KNOW what you are doing NOTE: Checked options will be applied if set before CI RunConfig/PrepareRunConfig step

Run these jobs only (required builds will be added automatically):

  • [ ] Integration Tests
  • [ ] Stateless tests
  • [ ] Stateful tests
  • [ ] Unit tests
  • [ ] Performance tests
  • [ ] All with aarch64
  • [ ] All with ASAN
  • [ ] All with TSAN
  • [ ] All with Analyzer
  • [ ] All with Azure
  • [ ] Add your option here

Deny these jobs:

  • [ ] Fast test
  • [ ] Integration Tests
  • [ ] Stateless tests
  • [ ] Stateful tests
  • [ ] Performance tests
  • [ ] All with ASAN
  • [ ] All with TSAN
  • [ ] All with MSAN
  • [ ] All with UBSAN
  • [ ] All with Coverage
  • [ ] All with Aarch64

Extra options:

  • [ ] do not test (only style check)
  • [ ] disable merge-commit (no merge from master before tests)
  • [ ] disable CI cache (job reuse)

Only specified batches in multi-batch jobs:

  • [ ] 1
  • [ ] 2
  • [ ] 3
  • [ ] 4

wudidapaopao avatar Sep 24 '25 07:09 wudidapaopao

Thanks, I didn't expect the topic to move so quickly

Out of curiosity, do you also intend not to require these dependencies in the pre-built wheel? https://github.com/chdb-io/chdb/blob/main/setup.py#L177

        setup(
            packages=["chdb"],
            version=versionStr,
            package_data={"chdb": pkg_files},
            exclude_package_data={"": ["*.pyc", "src/**"]},
            ext_modules=ext_modules,
            python_requires=">=3.8",
>           install_requires=[
>               "pyarrow>=13.0.0",
>               "pandas>=2.0.0",
            ],

matrey avatar Sep 25 '25 01:09 matrey

Thanks, I didn't expect the topic to move so quickly

Out of curiosity, do you also intend not to require these dependencies in the pre-built wheel? https://github.com/chdb-io/chdb/blob/main/setup.py#L177

        setup(
            packages=["chdb"],
            version=versionStr,
            package_data={"chdb": pkg_files},
            exclude_package_data={"": ["*.pyc", "src/**"]},
            ext_modules=ext_modules,
            python_requires=">=3.8",
>           install_requires=[
>               "pyarrow>=13.0.0",
>               "pandas>=2.0.0",
            ],

@wudidapaopao I think it's reasonable to remove pyarrow and pandas as necessary deps. And we should run our unit tests in 2 groups w/ and w/o pyarrow and pandas.

auxten avatar Sep 29 '25 07:09 auxten