beam icon indicating copy to clipboard operation
beam copied to clipboard

Update MongoDB driver to mongodb-driver-legacy:5.4.0

Open jyemin opened this issue 11 months ago • 15 comments

Upgrade MongoDB Driver to latest available release.

Note that as of 4.0 the mongo-java-driver uber jar is no longer published, so using mongodb-driver-legacy (which will pull in transitive dependencies) in its place.

I'll also note that I wasn't able to build the project locally without errors so I'm using the PR to check whether everything still works.

https://github.com/apache/beam/issues/32088


Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily:

  • [ ] Mention the appropriate issue in your description (for example: addresses #123), if applicable. This will automatically add a link to the pull request in the issue. If you would like the issue to automatically close on merging the pull request, comment fixes #<ISSUE NUMBER> instead.
  • [ ] Update CHANGES.md with noteworthy changes.
  • [ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

See the Contributor Guide for more tips on how to make review process smoother.

To check the build health, please visit https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md

GitHub Actions Tests Status (on master branch)

Build python source distribution and wheels Python tests Java tests Go tests

See CI.md for more information about GitHub Actions CI or the workflows README to see a list of phrases to trigger workflows.

jyemin avatar Feb 27 '25 14:02 jyemin

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

github-actions[bot] avatar Feb 27 '25 15:02 github-actions[bot]

assign set of reviewers

derrickaw avatar Mar 18 '25 03:03 derrickaw

Assigning reviewers. If you would like to opt out of this review, comment assign to next reviewer:

R: @robertwb for label java. R: @Abacn for label build. R: @johnjcasey for label io.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

The PR bot will only process comments in the main thread (not review comments).

github-actions[bot] avatar Mar 18 '25 03:03 github-actions[bot]

Apologies for not being able to get all the checks to pass. Would appreciate some assistance.

jyemin avatar Mar 18 '25 12:03 jyemin

The failed test is

org.junit.ComparisonFailure: expected:<{"$match": {"[_id": {"$gt": {"$oid": "52cc8f6254c5317943000005"}}]}}> but was:<{"$match": {"[$and": [{"_id": {"$gt": {"$oid": "52cc8f6254c5317943000005"}}}]]}}>
	at org.junit.Assert.assertEquals(Assert.java:117)

From failed test, go to "Summary", One can see "For more details on these failures, see this check" with a link, then the stack trace for failed test is there.

Also, there is "JUnit Test Results" uploaded in Artifact Section and one can download

Abacn avatar Mar 18 '25 15:03 Abacn

Reminder, please take a look at this pr: @robertwb @Abacn @johnjcasey

github-actions[bot] avatar Mar 26 '25 12:03 github-actions[bot]

The 5.4.0 driver was recently released, and I updated the dependency to reflect that. Note that 5.4.0, for the first time, includes a BOM. I'm unsure how to reference the BOM in this project, but perhaps maintainers can advise.

jyemin avatar Mar 26 '25 16:03 jyemin

Also unclear why this error is happening:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:analyzeClassesDependencies'.
> Dependency analysis found issues.
  usedUndeclaredArtifacts
   - org.mongodb:bson:5.4.0@jar
   - org.mongodb:mongodb-driver-core:5.4.0@jar
  unusedDeclaredArtifacts
   - org.mongodb:mongodb-driver-legacy:5.4.0@jar

Can anyone advise?

jyemin avatar Mar 26 '25 19:03 jyemin

Also unclear why this error is happening:

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':sdks:java:extensions:sql:analyzeClassesDependencies'.
> Dependency analysis found issues.
  usedUndeclaredArtifacts
   - org.mongodb:bson:5.4.0@jar
   - org.mongodb:mongodb-driver-core:5.4.0@jar
  unusedDeclaredArtifacts
   - org.mongodb:mongodb-driver-legacy:5.4.0@jar

Can anyone advise?

This is a gradle-dependency-analyze check. It suggests in ":sdks:java:extensions:sql" it is still mongodb-driver-core instead of mongodb-driver-legacy are actually used one

Abacn avatar Mar 26 '25 22:03 Abacn

@Abacn I tried various combinations of dependencies, but have not been able to find the right combination that works. I would appreciate if someone with more knowledge of the build system could give it a shot, but I understand if this PR is not top of mind for the maintainers.

jyemin avatar Apr 01 '25 21:04 jyemin

Reminder, please take a look at this pr: @robertwb @Abacn @johnjcasey

github-actions[bot] avatar Apr 09 '25 12:04 github-actions[bot]

Assigning new set of reviewers because Pr has gone too long without review. If you would like to opt out of this review, comment assign to next reviewer:

R: @kennknowles for label java. R: @damccorm for label build. R: @johnjcasey for label io.

Available commands:

  • stop reviewer notifications - opt out of the automated review tooling
  • remind me after tests pass - tag the comment author after tests pass
  • waiting on author - shift the attention set back to the author (any comment or push by the author will return the attention set to the reviewers)

github-actions[bot] avatar Apr 14 '25 12:04 github-actions[bot]

R: @Abacn (keeping sticky since Yi has started to review)

damccorm avatar Apr 14 '25 13:04 damccorm

Stopping reviewer notifications for this pull request: review requested by someone other than the bot, ceding control. If you'd like to restart, comment assign set of reviewers

github-actions[bot] avatar Apr 14 '25 13:04 github-actions[bot]

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

github-actions[bot] avatar Jun 14 '25 12:06 github-actions[bot]

@jyemin could you address the error and resolve the conflict so that the review can continue ?

Please let us know if you need additional help.

chamikaramj avatar Jun 20 '25 19:06 chamikaramj

waiting on author

derrickaw avatar Aug 06 '25 17:08 derrickaw

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

github-actions[bot] avatar Oct 06 '25 12:10 github-actions[bot]

This pull request has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

github-actions[bot] avatar Oct 13 '25 12:10 github-actions[bot]