[FEATURE]Move PPL AST (SQL) and consolidate
Is your feature request related to a problem?
This second step of consolidating the PPL spec into a single repository which will be deployed as an independent artifact.
What solution would you like?
Second step:
- Move PPL AST visitors into this repo
- consolidate different divergence between SQL's & Spark's PPL AST
- add as a jar artifact publishing to Maven
#23 - Main RFC
Do you have any additional context?
- https://github.com/opensearch-project/sql/tree/main/ppl/src/main/antlr
- https://github.com/opensearch-project/sql/tree/main/ppl/src/main/java/org/opensearch/sql/ppl
- https://github.com/opensearch-project/opensearch-spark/tree/main/ppl-spark-integration/src/main/java/org/opensearch/sql/ast
- https://github.com/opensearch-project/opensearch-spark/tree/main/ppl-spark-integration/src/main/antlr4
We should have some guidelines ahead of the implementation, regarding the part of consolidate different divergence between SQL's & Spark's PPL AST
Some divergence I foresee would be:
-
If the PPL command on both sides take the same amount of arguments but in different order, the order from
SQL Pluginshould be honoured. -
If arguments are being read differently in terms of type hierarchy, cast up should happen. (Number v.s Integer, Date represent in String format v.s Date)
@andy-k-improving I agree - we need to review each case and determine the differences and see :
- do they actually matter for the result ?
- what is the cause of this diff ?
- is it simple to change to code to overcome this diff OR add the additional
dialectlayer that would do the translation based on each command (thedialectwill have its own grammar file)
[Catch All Triage - 1, 2, 3]