spark-redshift icon indicating copy to clipboard operation
spark-redshift copied to clipboard

How to set wlm_query_slot_count

Open sylvinho81 opened this issue 7 years ago • 0 comments

I am using the spark-redshift connector in order to launch a query from Spark:

val results = spark.sqlContext.read
      .format("com.databricks.spark.redshift")
      .option("url", url_connection)
      .option("query", query)
      .option("aws_iam_role", iam_role)
      .option("tempdir", base_path_temp)
      .load()

I would like to increase the slot count in order to improve the query, because is disk-based. But I don't know how to do the next query in the connector:

set wlm_query_slot_count to 3;

I don't see how to do this , since in the read command the connector doesn't provide preactions and postactions like in the write command.

sylvinho81 avatar Sep 18 '18 15:09 sylvinho81