JDBC.NET icon indicating copy to clipboard operation
JDBC.NET copied to clipboard

Azure Databricks JDBC + Driver

Open Nerus87 opened this issue 1 year ago • 2 comments

Azure cloud Driver: DatabricksJDBC42.jar Download: https://www.databricks.com/spark/jdbc-drivers-download Guide: https://docs.databricks.com/en/_extras/documents/Databricks-JDBC-Driver-Install-and-Configuration-Guide.pdf

Hello, I have a problem fetching data, DB can be changed and table columns are shown but all the data are missing. I've tried even with Dapper but it is not working.

Nerus87 avatar Mar 20 '24 19:03 Nerus87

Additional info, logs from connection:

Mar 21 10:07:43.349 WARN  31 com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run: Caught Throwable in BackgroundFetcher.run() while the thread is running: null
Mar 21 10:07:43.354 ERROR 27 com.databricks.client.exceptions.ExceptionConverter.toSQLException: [Databricks][JDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 27. Background thread ID: 31. Error caught: null.
java.sql.SQLException: [Databricks][JDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 27. Background thread ID: 31. Error caught: null.
	at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
Caused by: com.databricks.client.support.exceptions.GeneralException: [Databricks][JDBCDriver](500540) Error caught in BackgroundFetcher. Foreground thread ID: 27. Background thread ID: 31. Error caught: null.
	... 5 more
Caused by: java.lang.ExceptionInInitializerError
	at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.getDirectBuffer(ArrowBuf.java:229)
	at com.databricks.client.jdbc42.internal.apache.arrow.memory.ArrowBuf.nioBuffer(ArrowBuf.java:224)
	at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.ReadChannel.readFully(ReadChannel.java:87)
	at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageSerializer.readMessageBody(MessageSerializer.java:728)
	at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.message.MessageChannelReader.readNext(MessageChannelReader.java:67)
	at com.databricks.client.jdbc42.internal.apache.arrow.vector.ipc.ArrowStreamReader.loadNextBatch(ArrowStreamReader.java:145)
	at com.databricks.client.spark.arrow.ArrowBuffer.handleInitializeBuffer(Unknown Source)
	at com.databricks.client.hivecommon.api.HiveServer2BaseBuffer.initializeBuffer(Unknown Source)
	at com.databricks.client.spark.jdbc.ResultFileDownloadManager.getNextDownloadedFile(Unknown Source)
	at com.databricks.client.spark.jdbc.DownloadableFetchClient.fetchNRows(Unknown Source)
	at com.databricks.client.hivecommon.api.HS2Client.fetchRows(Unknown Source)
	at com.databricks.client.hivecommon.dataengine.BackgroundFetcher.run(Unknown Source)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
	at java.base/java.util.concurrent.FutureTask.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: Failed to initialize MemoryUtil.
	at com.databricks.client.jdbc42.internal.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:136)
	... 17 more
Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make field long java.nio.Buffer.address accessible: module java.base does not "opens java.nio" to unnamed module @1c655221
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source)
	at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(Unknown Source)
	at java.base/java.lang.reflect.Field.checkCanSetAccessible(Unknown Source)
	at java.base/java.lang.reflect.Field.setAccessible(Unknown Source)
	at com.databricks.client.jdbc42.internal.apache.arrow.memory.util.MemoryUtil.<clinit>(MemoryUtil.java:84)
	... 17 more

Mar 21 10:07:43.392 WARN  27 com.databricks.client.hivecommon.utils.HeartBeatThread.removeOperationHandle: THandleIdentifier(guid:57 FB 64 B6 55 22 48 92 BF 3B 49 A6 7A D7 70 58, secret:20 7C 01 CB 74 64 42 24 99 E4 C5 1E 00 60 30 E6) is not in the heartbeat opeartion handle list
Mar 21 10:07:43.392 WARN  27 com.databricks.client.hivecommon.utils.HeartBeatThread.removeOperationHandle: THandleIdentifier(guid:57 FB 64 B6 55 22 48 92 BF 3B 49 A6 7A D7 70 58, secret:20 7C 01 CB 74 64 42 24 99 E4 C5 1E 00 60 30 E6) is not in the heartbeat opeartion handle list

Nerus87 avatar Mar 21 '24 09:03 Nerus87

I'm try clone the source and add this --add-opens java.base/java.nio=ALL_UNNAMED in the JdbcBridge before execute command java and it dose working. Waiting the author add this config to the java package. image

idunnoaiii avatar May 28 '24 12:05 idunnoaiii