Added SparkSQL support based on HiveDialect
SparkSQL is made to be almost compatible with HiveQL. However, the
show tables syntax for Spark SQL is slightly different. As such, a new
SparkSQL dialect is created based on the original HiveDialect with
get_table_names modified to accommodate the results returned from Spark SQL's
'SHOW TABLES'.
This fixes https://github.com/dropbox/PyHive/issues/150 without breaking the original Hive engine.
Any progress on this?
+1
+1 to get this PR merged :)
Do you thought also about changing get_view_names, get_table_names result based on isTemporary column ?
bump
@tanhaoqin this is failing the tests for some reason, maybe rebasing would fix the issue?
Changed some stuff so there's no longer any conflicts.
I am no longer working on the project that required this feature though. Didn't have time to write tests as I am not using any Hive/Presto on my side so the original Hive/Presto tests will fail anyway.
Many tests done by sqlalchemy_test_case will fail due the lack of support of some types by spark (SPARK-21529).
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.