hive icon indicating copy to clipboard operation
hive copied to clipboard

HIVE-28019 Fix query type information in proto files for load queries

Open ramesh0201 opened this issue 2 years ago • 6 comments

What changes were proposed in this pull request?

Query Type information for certain load queries are currently tagged as 'QUERY' operation type. As part of this change, load queries will be tagged as 'LOAD' query type.

Why are the changes needed?

Query Type information for load queries are not corrected tagged. This information needs to rightly set since it also gets logged to hive proto files, which in turn might get used to generate reports on the kind of queries that are being run.

Does this PR introduce any user-facing change?

Yes, log files will contain the query type field for load queries

Is the change a dependency upgrade?

No

How was this patch tested?

Most of the existing tests results that contain any load queries were modified righty and verified.

ramesh0201 avatar Jan 23 '24 10:01 ramesh0201

There are a lot of test files changed in the PR. Might be it is easier to verify the code/script that is used to update the test files.

niteshy avatar Mar 15 '24 18:03 niteshy

There are a lot of test files changed in the PR. Might be it is easier to verify the code/script that is used to update the test files.

The repo to test the changes are here https://github.infra.cloudera.com/rameshkumar/TestQueryTypes

ramesh0201 avatar Mar 19 '24 18:03 ramesh0201

Just FYI: https://issues.apache.org/jira/browse/HIVE-28128 https://issues.apache.org/jira/browse/HIVE-28129

Above are pending issues related to the query type information

ramesh0201 avatar Mar 19 '24 20:03 ramesh0201

I tend to lead towards keeping things as is for subjective things (unless there is a good reason to change). If we do end up changing the PRE/POST hook - it might make sense to do something like: console.printInfo("PREHOOK: type: " (isExplain ? "EXPLAIN " : "") + queryState.getCommandType(), false); So we get both bits of information - and as for the contents of the proto files maybe adding an isExplain? But as @zabetak pointed out it is pretty easy for people to do a prefix check on the query string (and it would be backwards compatible between versions of hive)

Just chiming in with my opinion - not sure what is the "right way" or even if there is a "right way".

jfsii avatar Apr 18 '24 21:04 jfsii

Quality Gate Passed Quality Gate passed

Issues
2 New issues
0 Accepted issues

Measures
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarCloud

sonarqubecloud[bot] avatar Apr 23 '24 14:04 sonarqubecloud[bot]

Thank you @zabetak and @jfsii for the review. I have modified this change to contain only the changes for the load queries. For the explain queries we will still fallback to the previous type information it had. Can you help review the latest patch?

ramesh0201 avatar Apr 23 '24 15:04 ramesh0201

This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the [email protected] list if the patch is in need of reviews.

github-actions[bot] avatar Jul 10 '24 00:07 github-actions[bot]