iceberg icon indicating copy to clipboard operation
iceberg copied to clipboard

Docs: Add build of particular compute engine versions

Open a49a opened this issue 3 years ago • 2 comments

Feature Request / Improvement

Add /gradlew build -DsparkVersions=3.2,3.3 -DflinkVersions=1.14 to the docs.

Query engine

No response

a49a avatar Aug 09 '22 03:08 a49a

Technically there's -DsparkVersions, -DflinkVersions, -DhiveVersions and -DscalaVersion, as we can see from dev/stage-binaries.sh.

So maybe we just document that each of those exists? Some things like scalaVersion only affect certain packages (namely Spark related ones).

For reference, the complete build steps at present (except the build command should be used not publishApachePublicationToMavenRepository):

https://github.com/apache/iceberg/blob/b1ab35d66c8f19c9107094b4e26aa8beae94bf22/dev/stage-binaries.sh#L21-L31

kbendick avatar Aug 09 '22 16:08 kbendick

Ideally this could be documented either:

  1. generically, so we don't have to update it - possibly linking to the dev/stage-binaries.sh script to show the complete set of options (though this could create confusion).
  2. pulled from the repository somehow so that the versioned docs always have the complete set of commands. But we'd want to emphasize when is it important to use certain ones (like the scala version only affects Spark 3.2 and 3.3).

If you want to try this one out @deadwind4, I don't think you'll hear any objections. I would cc @samredai in case he has any concerns or existing knowledge of anything on going related to this 🙂

kbendick avatar Aug 09 '22 18:08 kbendick

This issue has been automatically marked as stale because it has been open for 180 days with no activity. It will be closed in next 14 days if no further activity occurs. To permanently prevent this issue from being considered stale, add the label 'not-stale', but commenting on the issue is preferred when possible.

github-actions[bot] avatar Feb 06 '23 00:02 github-actions[bot]

This issue has been closed because it has not received any activity in the last 14 days since being marked as 'stale'

github-actions[bot] avatar Feb 20 '23 00:02 github-actions[bot]