Support executing queries without pyarrow when it's missing
Fix https://github.com/chdb-io/chdb/issues/386
Changelog category (leave one):
- New Feature
Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):
...
Documentation entry for user-facing changes
- [ ] Documentation is written (mandatory for new features)
Information about CI checks: https://clickhouse.com/docs/en/development/continuous-integration/
CI Settings
NOTE: If your merge the PR with modified CI you MUST KNOW what you are doing NOTE: Checked options will be applied if set before CI RunConfig/PrepareRunConfig step
Run these jobs only (required builds will be added automatically):
- [ ] Integration Tests
- [ ] Stateless tests
- [ ] Stateful tests
- [ ] Unit tests
- [ ] Performance tests
- [ ] All with aarch64
- [ ] All with ASAN
- [ ] All with TSAN
- [ ] All with Analyzer
- [ ] All with Azure
- [ ] Add your option here
Deny these jobs:
- [ ] Fast test
- [ ] Integration Tests
- [ ] Stateless tests
- [ ] Stateful tests
- [ ] Performance tests
- [ ] All with ASAN
- [ ] All with TSAN
- [ ] All with MSAN
- [ ] All with UBSAN
- [ ] All with Coverage
- [ ] All with Aarch64
Extra options:
- [ ] do not test (only style check)
- [ ] disable merge-commit (no merge from master before tests)
- [ ] disable CI cache (job reuse)
Only specified batches in multi-batch jobs:
- [ ] 1
- [ ] 2
- [ ] 3
- [ ] 4
Thanks, I didn't expect the topic to move so quickly
Out of curiosity, do you also intend not to require these dependencies in the pre-built wheel? https://github.com/chdb-io/chdb/blob/main/setup.py#L177
setup(
packages=["chdb"],
version=versionStr,
package_data={"chdb": pkg_files},
exclude_package_data={"": ["*.pyc", "src/**"]},
ext_modules=ext_modules,
python_requires=">=3.8",
> install_requires=[
> "pyarrow>=13.0.0",
> "pandas>=2.0.0",
],
Thanks, I didn't expect the topic to move so quickly
Out of curiosity, do you also intend not to require these dependencies in the pre-built wheel? https://github.com/chdb-io/chdb/blob/main/setup.py#L177
setup( packages=["chdb"], version=versionStr, package_data={"chdb": pkg_files}, exclude_package_data={"": ["*.pyc", "src/**"]}, ext_modules=ext_modules, python_requires=">=3.8", > install_requires=[ > "pyarrow>=13.0.0", > "pandas>=2.0.0", ],
@wudidapaopao I think it's reasonable to remove pyarrow and pandas as necessary deps. And we should run our unit tests in 2 groups w/ and w/o pyarrow and pandas.