HIVE-12930: Support SSL Shuffle for LLAP
What changes were proposed in this pull request?
Added support for enabling SSL for LLAP shuffle. Made changes in LLAP Shuffle handler similar to Tez shuffle handler. Added ssl-server.xml configuration file to LLAP daemon configs since LLAP daemon acts as both server and client for shuffle.
Why are the changes needed?
These changes are needed to enable SSL for LLAP shuffle.
Does this PR introduce any user-facing change?
Yes, this change introduces hive.llap.shuffle.ssl.enabled configuration property for enabling SSL for LLAP shuffle. Both tez.runtime.shuffle.ssl.enable and hive.llap.shuffle.ssl.enabled have to be enabled for enabling SSL for LLAP shuffle. This is because the shuffle fetcher implementation in Tez is dependent on tez.runtime.shuffle.ssl.enable for enabling SSL while communicating with shuffle handler for fetching shuffle data.
Is the change a dependency upgrade?
No
How was this patch tested?
Manually verified that SSL is enabled for the LLAP shuffle.
Quality Gate passed
Issues
0 New issues
0 Accepted issues
Measures
0 Security Hotspots
No data about Coverage
No data about Duplication
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.
LGTM, but I would ask for @abstractdog opinion
- can we fix the "SUFFLE" typos if we're already touching this part?
- can you please share more details about the manual testing? real cluster vs. some mini llap cluster? Can this happen with a MiniLlapCluster as a test?
tez has TestSecureShuffle for this, which would be hard to adapt, but hive's own mini llap based tests make sense to me, as this is a new feature (at least in hive) consider use tez unit test for guidance with setting up SSL: https://github.com/apache/tez/blob/30e9cdbff7adb75c3553d5cbb8515761a7231e07/tez-tests/src/test/java/org/apache/tez/test/TestSecureShuffle.java#L241
This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.