hive icon indicating copy to clipboard operation
hive copied to clipboard
verified publisher icon apache

HIVE-23745: Avoid copying userpayload in task communicator

Open difin opened this issue 3 years ago • 1 comments

What changes were proposed in this pull request?

Copy of user payload take a few milliseconds sometimes. This delay adds up for all tasks of a single vertex in LlapTaskCommunicator as it processes tasks one by one. User payload never changes in this codepath. Copy used to be made because of limitations of Protobuf library. Protobuf added a UnsafeByteOperations class that avoid copying of ByteBuffers in 3.1 version. This can be resolved now since Protobuf has been upgraded.

Why are the changes needed?

To reduce query startup latency by removing unneeded copying of ByteBuffers.

Does this PR introduce any user-facing change?

No

How was this patch tested?

Unit and integration tests

difin avatar Sep 22 '22 15:09 difin

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug C 1 Bug
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 4 Code Smells

No Coverage information No Coverage information
No Duplication information No Duplication information

sonarqubecloud[bot] avatar Sep 22 '22 15:09 sonarqubecloud[bot]

Thanks, @abstractdog!

difin avatar Oct 05 '22 17:10 difin