Raj Patel
Raj Patel
1. Serverless dataproc version migration from `1.1` to `1.2` 2. Maintained maven compile version to JDK11 3. Change bigquery format to more enterprise standard 4. Upgrade existing libraries to match...
Currently we are using export JARS option to provide spark bigquery connector jar file. It is available through maven central [repository](https://mvnrepository.com/artifact/com.google.cloud.spark/spark-bigquery-connector-common). Update pom.xml file to be available from there.
Our spark bigquery connector is supporting append and overwrite mode based on [documentation](https://github.com/GoogleCloudDataproc/spark-bigquery-connector?tab=readme-ov-file#writing-data-to-bigquery). We need to thoroughly test and validate various scenarios and update README.md file for writing to BigQuery...