openhouse icon indicating copy to clipboard operation
openhouse copied to clipboard

Set spark.driver.host to localhost in integration tests

Open teamurko opened this issue 11 months ago • 0 comments

Summary

Started getting binding failure because hostname has changed from 127.0.0.1 to some internal network ip. spark.driver.bindAddress defaults to {spark.driver.host}. It's a common practice to set driver host to localhost in tests.

Can't assign requested address: Service 'sparkDriver' failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service 'sparkDriver' (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.
java.net.BindException: Can't assign requested address: Service 'sparkDriver' failed after 16 retries (on a random free port)! Consider explicitly setting the appropriate binding address for the service 'sparkDriver' (for example spark.driver.bindAddress for SparkDriver) to the correct binding address.

Changes

  • [ ] Client-facing API Changes
  • [ ] Internal API Changes
  • [ ] Bug Fixes
  • [ ] New Features
  • [ ] Performance Improvements
  • [ ] Code Style
  • [ ] Refactoring
  • [ ] Documentation
  • [x] Tests

For all the boxes checked, please include additional details of the changes made in this pull request.

Testing Done

  • [ ] Manually Tested on local docker setup. Please include commands ran, and their output.
  • [ ] Added new tests for the changes made.
  • [x] Updated existing tests to reflect the changes made.
  • [ ] No tests added or updated. Please explain why. If unsure, please feel free to ask for help.
  • [ ] Some other form of testing like staging or soak time in production. Please explain.

For all the boxes checked, include a detailed description of the testing done for the changes made in this pull request.

Additional Information

  • [ ] Breaking Changes
  • [ ] Deprecations
  • [ ] Large PR broken into smaller PRs, and PR plan linked in the description.

For all the boxes checked, include additional details of the changes made in this pull request.

teamurko avatar Feb 14 '25 02:02 teamurko