Docs: Add build of particular compute engine versions
Feature Request / Improvement
Add /gradlew build -DsparkVersions=3.2,3.3 -DflinkVersions=1.14 to the docs.
Query engine
No response
Technically there's -DsparkVersions, -DflinkVersions, -DhiveVersions and -DscalaVersion, as we can see from dev/stage-binaries.sh.
So maybe we just document that each of those exists? Some things like scalaVersion only affect certain packages (namely Spark related ones).
For reference, the complete build steps at present (except the build command should be used not publishApachePublicationToMavenRepository):
https://github.com/apache/iceberg/blob/b1ab35d66c8f19c9107094b4e26aa8beae94bf22/dev/stage-binaries.sh#L21-L31
Ideally this could be documented either:
- generically, so we don't have to update it - possibly linking to the
dev/stage-binaries.shscript to show the complete set of options (though this could create confusion). - pulled from the repository somehow so that the versioned docs always have the complete set of commands. But we'd want to emphasize when is it important to use certain ones (like the scala version only affects Spark 3.2 and 3.3).
If you want to try this one out @deadwind4, I don't think you'll hear any objections. I would cc @samredai in case he has any concerns or existing knowledge of anything on going related to this 🙂
This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.
This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'