Speed up the Hudi test by caching a downloaded file
What does this PR do?
Hudi tests can sometimes take more than 30 minutes because downloading the Spark archive from https://archive.apache.org/ is slow. This PR introduces caching for that download at the GitHub Actions level.
The cache can also be reused by other integrations. It’s configured via a .github-cache.yml file located in each integration directory.
The .github-cache.yml file defines a list of files (with paths relative to the integrations directory) and their associated download URLs.
GitHub Actions uses this configuration to cache and restore the files, which helps with slow or unreliable download sources.
The cache has no impact on tests running locally on the developer's environment.
Motivation
Speed up CI.
Review checklist (to be filled by reviewers)
- [ ] Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
- [ ] Add the
qa/skip-qalabel if the PR doesn't need to be tested during QA. - [ ] If you need to backport this PR to another branch, you can add the
backport/<branch-name>label to the PR and it will automatically open a backport PR once this one is merged
Codecov Report
:white_check_mark: All modified and coverable lines are covered by tests.
:white_check_mark: Project coverage is 88.99%. Comparing base (46169ca) to head (82c6fbf).
:warning: Report is 40 commits behind head on master.
Additional details and impacted files
:rocket: New features to boost your workflow:
- :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
- :package: JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.
⚠️ The qa/skip-qa label has been added with shippable changes
The following files, which will be shipped with the agent, were modified in this PR and
the qa/skip-qa label has been added.
You can ignore this if you are sure the changes in this PR do not require QA. Otherwise, consider removing the label.
List of modified files that will be shipped with the agent
hudi/hatch.toml
This PR doesn't actually touch files owned by windows-products. Windows-products was added in the reviewer list when I added some temporary testing code to check Windows compatibility.