openhouse icon indicating copy to clipboard operation
openhouse copied to clipboard

Introduce branching and support for spark.wap.branch

Open cbb330 opened this issue 4 months ago • 0 comments

Summary

Implement Iceberg table branching functionality in OpenHouse internal catalog with comprehensive testing infrastructure.

The old code was getting unwieldy with nested for-loops, mutable collections, and scattered state management, repeated logic and overall needed a restructure with branching supported first-class. i feel more confident that this is reliable.

Changes

  • Branch Operations: Added branch management methods to OpenHouseInternalTableOperations including create, delete, list, and commit reference handling
  • Multi-branch Commit Support: Implemented logic to handle commits across different branch references with checks to reject ambiguous commits
  • WAP Branch Integration: Added Write-Audit-Publish (WAP) branch support with proper test coverage
  • Test Infrastructure: Created comprehensive test suite (BranchTestSpark3_5) covering branch operations, edge cases, and integration scenarios

Changes

  • [X] Client-facing API Changes
  • [X] Internal API Changes
  • [ ] Bug Fixes
  • [X] New Features
  • [ ] Performance Improvements
  • [ ] Code Style
  • [X] Refactoring
  • [ ] Documentation
  • [X] Tests

Testing Done

  • [ ] Manually Tested on local docker setup. Please include commands ran, and their output.
  • [X] Added new tests for the changes made.
  • [X] Updated existing tests to reflect the changes made.
  • [ ] No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
  • [ ] Some other form of testing like staging or soak time in production. Please explain.

Additional Information

  • [ ] Breaking Changes
  • [ ] Deprecations
  • [ ] Large PR broken into smaller PRs, and PR plan linked in the description.

cbb330 avatar Sep 29 '25 19:09 cbb330