PARQUET-1711: support recursive proto schemas by limiting recursion depth
- This is an alternative approach to supporting recursion to apache#445 and apache#988.
- This approach could address the other recursion related issues (PARQUET-129, PARQUET-554).
- TODO: ReadSupport
Jira
- [x] My PR addresses the following Parquet Jira issues and references them in the PR title. For example, "PARQUET-1234: My Parquet PR"
- https://issues.apache.org/jira/browse/PARQUET-1711
- In case you are adding a dependency, check if the license complies with the ASF 3rd Party License Policy.
Tests
- [x] My PR adds the following unit tests OR does not need testing for this extremely good reason:
-
ProtoSchemaConverterTest#test*Recursion -
ProtoWriteSupportTest#test*Recursion
-
Commits
- [x] My commits all reference Jira issues in their subject lines. In addition, my commits follow the guidelines from "How to write a good git commit message":
- Subject is separated from body by a blank line
- Subject is limited to 50 characters (not including Jira issue reference)
- Subject does not end with a period
- Subject uses the imperative mood ("add", not "adding")
- Body wraps at 72 characters
- Body explains "what" and "why", not "how"
Documentation
- [x] In case of new functionality, my PR adds documentation that describes how to use it.
- All the public functions and the classes in the PR contain Javadoc that explain what it does
ping
fixed missing dep issue. can someone approve the ci flow?
thanks for the review. updated to handle the logging perf concern as well as fixing the javadoc errors.
ping
Mostly looks reasonable, I'm not too familiar with parquet-mr @shangxinli can you recommend someone who might be able to give a better review?
Mostly looks reasonable, I'm not too familiar with parquet-mr @shangxinli can you recommend someone who might be able to give a better review?
pinging @shangxinli :)
@ggershinsky Can you have a look?
I would also like to recommend adding @matthieun as a co-author to this PR, per the discussion in the parallel PR.
I would also like to recommend adding @matthieun as a co-author to this PR, per the discussion in the parallel PR.
~how do you do this?~ i think i did it right...
can someone retry the github actions? there seemed to have been a transient issue that caused one of the test/build targets to fail. i'd like to get this change in this week.
@ggershinsky what is the process to merge this? Does parquet-mr just use the github UI?
yep, just the squash/merge button.
@ggershinsky
i'd love to just hit the button. i don't see it. the workflow for travis ci had a failure due to a transient connection issue, and so it wasn't giving me the option to merge. the ui messaging also states that "Only those with write access to this repository can merge pull requests."
@shangxinli are you ok with this PR in its current form?
yeah, i still don't see a button to merge. it now shows everything approved, checks passed, and no conflicts.
i think a committer needs to merge.
@jinyius only committers can see the button. I was asking because different repos have different commit procedures. Should be able to merge this soon as long as @shangxinli doesn't express concerns.
LGTM