utPLSQL-cli icon indicating copy to clipboard operation
utPLSQL-cli copied to clipboard

Detected Oracle Driver stuck during Statement initialization

Open DrErique opened this issue 6 years ago • 45 comments

Hello,

I get the error message above regularly after two or three succesfull testings. Then I have two or three unsuccesful tries with the Oracle Driver stucks, after that in the next try can be succesfull again. There is no unpredictability if the driver stucks or not. So we have

30-Apr-2019 15:20:52 30-Apr-2019 15:20:52 C:\Users\vmwareadmin\bamboo-agent-home\xml-data\build-dir\SWITEGIT-WT-JOB1>utplsql run UNITTEST/P6YkF-k4il83_0s@AGTEST -p=UNITTEST.UT_PKG_AUSW_HP_SCHAEDEN -f=UT_JUNIT_REPORTER -o=C:\Users\vmwareadmin\bamboo-agent-home\xml-data\build-dir\SWITEGIT-WT-JOB1\UNIT_TEST_AGTEST_30-04-2019.xml 30-Apr-2019 15:20:52 ###################### utPLSQL cli ####################### 30-Apr-2019 15:20:52 # # 30-Apr-2019 15:20:52 # utPLSQL-cli 3.1.6.local # 30-Apr-2019 15:20:52 # utPLSQL-java-api 3.1.6.411 # 30-Apr-2019 15:20:52 # Java-Version: 1.8.0_201 # 30-Apr-2019 15:20:52 # ORACLE_HOME: C:\oracle\product\12.2.0\client_1_x64 # 30-Apr-2019 15:20:52 # NLS_LANG: null # 30-Apr-2019 15:20:52 # # 30-Apr-2019 15:20:52 # Thanks for testing! # 30-Apr-2019 15:20:52 # # 30-Apr-2019 15:20:52 ########################################################## 30-Apr-2019 15:20:52 30-Apr-2019 15:20:53 HikariPool-1 - Driver does not support get/set network timeout for connections. (Nicht unterstützte Funktion: getNetworkTimeout) 30-Apr-2019 15:20:53 Use connection string jdbc:oracle:oci8:/@AGTEST 30-Apr-2019 15:20:53 Successfully connected to database. UtPLSQL core: v3.1.2.2134 30-Apr-2019 15:20:53 Oracle-Version: 12.2.0.1.0 30-Apr-2019 15:20:53 Running tests now. 30-Apr-2019 15:20:53 -------------------------------------- 30-Apr-2019 15:20:53 TestRunner initialized 30-Apr-2019 15:20:53 Running on utPLSQL v3.1.2.2134 30-Apr-2019 15:20:53 Initializing reporters 30-Apr-2019 15:20:55 Detected Oracle driver stuck during Statement initialization

I tried to experiment with different Clients and Java Versions but as it is said here in the other issues, there is not much scope for experiments while we have to use Oracle Client 12.2 with jdbc 8 for the 3.1.6 Version of the utPLSQL-cli. Since it's actually working and it's also the correct configuration I wonder what I can do to become more stability in the testings. Perhaps I can send you a debug file of a failed test or you can suggest some better working versions of the ojdbc8.jar or the orai18n.jar ( both from 22.02.2019 ) ?

Greetings from Hannover, Germany !

DrErique avatar Apr 30 '19 13:04 DrErique

Hi @DrErique

what happens after the Detected Oracle driver stuck during Statement initialization? The expected behavior would be that CLI does an automatic retry (and potentially some error stacks are written). Does this happen? It's a problem we investigate since several months now and our solution (because it is very likely to be a driver bug) was to have an internal watchdog which detects such stuck scenario and does up to 5 retries. In that case the outcome of your tests wouldn't be changed, the only thing that happens is that the previous connection is abandoned and a second one is started.

Also: what would be your expectation how CLI deals with such a scenario?

Cheers, Samuel

P.S. If you happen to be at ApexConnect in Bonn next week, make sure to say hello :)

pesse avatar Apr 30 '19 14:04 pesse

Hello Samuel,

Thanks for the quick response.

As you probably have noticed the utPLSQL Command starts from a Bamboo – Powershell – Skript – Task.

I do not notice any retry .

Last entry in the Log is "Detected Oracle Driver stuck …" and the build runs endless until it is stopped manually. As you can see in the picture the build ran for 21 minutes.

Maybe I have to set some special parameters for the retries ?

As you already found out retrying the connection for 1 – n times is the only thing you can offer in such cases. But in my case ist seems there is nor real retrying happening ….

[cid:[email protected]]

Mit freundlichen Grüßen

Derik Wehrmann


Tel: -551 | Test- und Qualitätsmanager | IT SCM und Support

Von: Samuel Nitsche [email protected] Gesendet: Dienstag, 30. April 2019 16:04 An: utPLSQL/utPLSQL-cli [email protected] Cc: Wehrmann, Derik [email protected]; Mention [email protected] Betreff: Re: [utPLSQL/utPLSQL-cli] Detected Oracle Driver stuck during Statement initialization (#145)

Hi @DrEriquehttps://github.com/DrErique

what happens after the Detected Oracle driver stuck during Statement initialization? The expected behavior would be that CLI does an automatic retry (and potentially some error stacks are written). Does this happen? It's a problem we investigate since several months now and our solution (because it is very likely to be a driver bug) was to have an internal watchdog which detects such stuck scenario and does up to 5 retries. In that case the outcome of your tests wouldn't be changed, the only thing that happens is that the previous connection is abandoned and a second one is started.

Also: what would be your expectation how CLI deals with such a scenario?

Cheers, Samuel

P.S. If you happen to be at ApexConnect in Bonn next week, make sure to say hello :)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/utPLSQL/utPLSQL-cli/issues/145#issuecomment-487965097, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIPTY43SEIDGFHTFQWDVQA3PTBGWPANCNFSM4HJLZYZA.

DrErique avatar Apr 30 '19 15:04 DrErique

Hm, this is strange - the Stuck is detected but it's not aborted (for whatever reason). Can you try a different OJDBC driver? I use ojdbc8-12.2.0.1.jar currently.

pesse avatar May 03 '19 08:05 pesse

Hello Samuel,

I'll get that one and we see if this works better next week.

Have a nice weekend !

Mit freundlichen Grüßen

Derik Wehrmann


Tel: -551 | Test- und Releasemanager | IT SCM

Von: Samuel Nitsche [email protected] Gesendet: Freitag, 3. Mai 2019 10:55 An: utPLSQL/utPLSQL-cli [email protected] Cc: Wehrmann, Derik [email protected]; Mention [email protected] Betreff: Re: [utPLSQL/utPLSQL-cli] Detected Oracle Driver stuck during Statement initialization (#145)

Hm, this is strange - the Stuck is detected but it's not aborted (for whatever reason). Can you try a different OJDBC driver? I use ojdbc8-12.2.0.1.jar currently.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/utPLSQL/utPLSQL-cli/issues/145#issuecomment-489016702, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIPTY44UZUODUCVDHWFEV4TPTP4UPANCNFSM4HJLZYZA.

DrErique avatar May 03 '19 09:05 DrErique

Hello Samuel,

Unfortunately that didn't solve the problem. Do we have any debug option ?

You mentioned the –d Parameter in another issue ?

What about this "HikariPool-1 - Driver does not support get/set network timeout for connections. " report. Perhaps we have a problem with the HikariCP ?

DrErique avatar May 08 '19 14:05 DrErique

Yeah, this line is strange - didn't notice it so far in any of my runs. Did you use the latest version of cli from the github page? (yes, there's still something wrong with putting the build number into the version automatically)

pesse avatar May 09 '19 07:05 pesse

I'll try that also, downloading latest version. We see if this works better … ☺

Von: Samuel Nitsche [email protected] Gesendet: Donnerstag, 9. Mai 2019 09:24 An: utPLSQL/utPLSQL-cli [email protected] Cc: Wehrmann, Derik [email protected]; Mention [email protected] Betreff: Re: [utPLSQL/utPLSQL-cli] Detected Oracle Driver stuck during Statement initialization (#145)

Yeah, this line is strange - didn't notice it so far in any of my runs. Did you use the latest version of cli from the github page? (yes, there's still something wrong with putting the build number into the version automatically)

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/utPLSQL/utPLSQL-cli/issues/145#issuecomment-490778064, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIPTY426DNC4Z4JRGNBDL4TPUPGQ5ANCNFSM4HJLZYZA.

DrErique avatar May 09 '19 12:05 DrErique

Hi, I am also facing same issue here with Oracle DB 11.2,Java-Version: 1.8.0_212 , latest utPLSQL-cli 3.1.6 and jdbc 6 (the one from oracle site)

Actually I wanted to file a request for allowing to specifying more HikariConfig options, i.e. addDataSourceProperty (e.g. allowPoolSuspension)...maybe it would help slightly to debug the problem ("enterprise configurations")

I should also add that utPLSQL 3.1.2 was working well (as expected) - Hikari update ....

2019-05-10 11:03:22 [main] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Starting...
2019-05-10 11:03:22 [main] DEBUG c.z.hikari.util.DriverDataSource - Loaded driver with class name oracle.jdbc.OracleDriver for jdbcUrl=jdbc:oracle:thin:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2019-05-10 11:03:23 [main] INFO  com.zaxxer.hikari.pool.PoolBase - HikariPool-1 - Driver does not support get/set network timeout for connections. (oracle.jdbc.driver.T4CConnection.getNetworkTimeout()I)
2019-05-10 11:03:23 [main] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection oracle.jdbc.driver.T4CConnection@2d928643
2019-05-10 11:03:23 [main] INFO  com.zaxxer.hikari.HikariDataSource - HikariPool-1 - Start completed.
2019-05-10 11:03:23 [main] INFO  o.u.c.d.TestedDataSourceProvider - Use connection string jdbc:oracle:thin:****/****@//xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
2019-05-10 11:03:23 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Pool stats (total=1, active=1, idle=0, waiting=0)
2019-05-10 11:03:23 [main] INFO  org.utplsql.cli.RunCommand - Successfully connected to database. UtPLSQL core: v3.1.1.1868
2019-05-10 11:03:23 [main] INFO  org.utplsql.cli.RunCommand - Oracle-Version: 11.2.0.4.0
2019-05-10 11:03:23 [main] DEBUG c.zaxxer.hikari.pool.ProxyConnection - HikariPool-1 - Executed rollback on connection oracle.jdbc.driver.T4CConnection@2d928643 due to dirty commit state on close().
2019-05-10 11:03:23 [main] DEBUG org.utplsql.api.reporter.Reporter - Database-reporter initialized, Type: UT_COVERAGE_HTML_REPORTER, ID: 8885D6BC54DF5E4BE05311C8130A76B7
2019-05-10 11:03:23 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection oracle.jdbc.driver.T4CConnection@483c7db7
2019-05-10 11:03:23 [main] DEBUG org.utplsql.api.reporter.Reporter - Database-reporter initialized, Type: UT_DOCUMENTATION_REPORTER, ID: 8885D6BC54E35E4BE05311C8130A76B7
2019-05-10 11:03:23 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection oracle.jdbc.driver.T4CConnection@46fb8ac0
2019-05-10 11:03:23 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Added connection oracle.jdbc.driver.T4CConnection@42c416c7
2019-05-10 11:03:23 [HikariPool-1 connection adder] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - After adding stats (total=4, active=0, idle=4, waiting=0)
2019-05-10 11:03:25 [pool-1-thread-1] INFO  org.utplsql.cli.RunTestRunnerTask - Running tests now.
2019-05-10 11:03:25 [pool-1-thread-1] INFO  org.utplsql.cli.RunTestRunnerTask - --------------------------------------
2019-05-10 11:03:25 [pool-1-thread-1] INFO  org.utplsql.api.TestRunner - TestRunner initialized
2019-05-10 11:03:25 [pool-1-thread-1] INFO  org.utplsql.api.TestRunner - Running on utPLSQL v3.1.1.1868
2019-05-10 11:03:25 [pool-1-thread-1] INFO  org.utplsql.api.TestRunner - Initializing reporters
2019-05-10 11:03:27 [pool-1-thread-1] ERROR org.utplsql.api.TestRunner - Detected Oracle driver stuck during Statement initialization
2019-05-10 11:03:31 [pool-1-thread-1] DEBUG c.zaxxer.hikari.pool.ProxyConnection - HikariPool-1 - Executed rollback on connection oracle.jdbc.driver.T4CConnection@2d928643 due to dirty commit state on close().
2019-05-10 11:03:53 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Pool stats (total=4, active=2, idle=2, waiting=0)
2019-05-10 11:04:23 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Pool stats (total=4, active=2, idle=2, waiting=0)
2019-05-10 11:04:53 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Pool stats (total=4, active=2, idle=2, waiting=0)
2019-05-10 11:05:23 [HikariPool-1 housekeeper] DEBUG com.zaxxer.hikari.pool.HikariPool - HikariPool-1 - Pool stats (total=4, active=2, idle=2, waiting=0)

EDIT: so using jdbc8 --the one for 12c on the database 11.2; instead of jdbc6 for 11.2), error has changed to .... This is also retried 4 times. I wonder if it has really something to do with Hikari, e.g https://github.com/brettwooldridge/HikariCP/issues/198

2019-05-10 12:51:32 [pool-1-thread-1] ERROR org.utplsql.api.TestRunner - Detected Oracle driver stuck during Statement initialization
2019-05-10 12:51:32 [pool-2-thread-1] WARN  c.zaxxer.hikari.pool.ProxyConnection - HikariPool-1 - **Connection oracle.jdbc.driver.T4CConnection@3b088d51 marked as broken because of SQLSTATE(08006)**, ErrorCode(17002)
java.sql.SQLRecoverableException: IO Error: Socket read interrupted
	at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:929)
	at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1119)
	at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3780)
	at oracle.jdbc.driver.T4CCallableStatement.executeInternal(T4CCallableStatement.java:1300)
	at oracle.jdbc.driver.OraclePreparedStatement.execute(OraclePreparedStatement.java:3887)
	at oracle.jdbc.driver.OracleCallableStatement.execute(OracleCallableStatement.java:4230)
	at oracle.jdbc.driver.OraclePreparedStatementWrapper.execute(OraclePreparedStatementWrapper.java:1079)
	at com.zaxxer.hikari.pool.ProxyPreparedStatement.execute(ProxyPreparedStatement.java:44)
	at com.zaxxer.hikari.pool.HikariProxyCallableStatement.execute(HikariProxyCallableStatement.java)
	at org.utplsql.api.FileMapper.buildFileMappingArray(FileMapper.java:84)
	at org.utplsql.api.FileMapper.buildFileMappingList(FileMapper.java:90)
	at org.utplsql.api.testRunner.AbstractTestRunnerStatement.createStatement(AbstractTestRunnerStatement.java:60)
	at org.utplsql.api.testRunner.AbstractTestRunnerStatement.<init>(AbstractTestRunnerStatement.java:33)
	at org.utplsql.api.testRunner.Pre312TestRunnerStatement.<init>(Pre312TestRunnerStatement.java:17)
	at org.utplsql.api.testRunner.TestRunnerStatementProvider.getCompatibleTestRunnerStatement(TestRunnerStatementProvider.java:37)
	at org.utplsql.api.compatibility.CompatibilityProxy.getTestRunnerStatement(CompatibilityProxy.java:157)
	at org.utplsql.api.TestRunner.lambda$initStatementWithTimeout$0(TestRunner.java:199)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.SocketTimeoutException: Socket read interrupted
	at oracle.net.nt.TimeoutSocketChannel.read(TimeoutSocketChannel.java:152)
	at oracle.net.ns.NIOHeader.readHeaderBuffer(NIOHeader.java:82)
	at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:139)
	at oracle.net.ns.NIOPacket.readFromSocketChannel(NIOPacket.java:101)
	at oracle.net.ns.NIONSDataChannel.readDataFromSocketChannel(NIONSDataChannel.java:80)
	at oracle.jdbc.driver.T4CMAREngineNIO.prepareForReading(T4CMAREngineNIO.java:98)
	at oracle.jdbc.driver.T4CMAREngineNIO.unmarshalUB1(T4CMAREngineNIO.java:534)
	at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:485)
	at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:252)
	at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:612)
	at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:223)
	at oracle.jdbc.driver.T4CCallableStatement.doOall8(T4CCallableStatement.java:56)
	at oracle.jdbc.driver.T4CCallableStatement.executeForRows(T4CCallableStatement.java:907)
	... 20 common frames omitted
2019-05-10 12:51:32 [pool-1-thread-3] WARN  c.zaxxer.hikari.pool.ProxyConnection - HikariPool-1 - Connection oracle.jdbc.driver.T4CConnection@1fb473a8 marked as broken because of SQLSTATE(08003), ErrorCode(17008)
java.sql.SQLRecoverableException: Closed Connection
	at oracle.jdbc.driver.PhysicalConnection.clearWarnings(PhysicalConnection.java:2573)
	at com.zaxxer.hikari.pool.ProxyConnection.close(ProxyConnection.java:246)
	at org.utplsql.cli.ReporterManager.lambda$startReporterGatherers$0(ReporterManager.java:130)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
2019-05-10 12:51:32 [pool-1-thread-2] WARN  c.zaxxer.hikari.pool.ProxyConnection - HikariPool-1 - Connection oracle.jdbc.driver.T4CConnection@742dfdaf marked as broken because of SQLSTATE(08003), ErrorCode(17008)
java.sql.SQLRecoverableException: Closed Connection
	at oracle.jdbc.driver.PhysicalConnection.clearWarnings(PhysicalConnection.java:2573)
	at com.zaxxer.hikari.pool.ProxyConnection.close(ProxyConnection.java:246)
	at org.utplsql.cli.ReporterManager.lambda$startReporterGatherers$0(ReporterManager.java:130)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

dmpe avatar May 10 '19 09:05 dmpe

For we don't necessarily need a connection pool, I'll try to remove HikariCP completely.

pesse avatar May 13 '19 07:05 pesse

ok, that sounds awesome if it can be done. Actually I wanted to build several versions of utplsql-cli with different HakariCP and jdbc drivers to test it out where the problem may be. i would try to report my results on this friday...though do no wait on me, just go ahead.

EDIT: @pesse ok. so i have just tried 2 recent versions of utPLSQL with different versions of Hakari and jdbc

For utPLSQL 3.1.6 code base:

Hakari 2.7.9 and 3.3.1 did not work

For utPLSQL 3.1.2 code base:

Hakari 2.7.9 and 3.3.1 did not work what works is only 2.7.2 version with this codebase, with jdbc7.

dmpe avatar May 13 '19 18:05 dmpe

I had a similar issue:

UtPLSQL core: v3.1.6.2735 Oracle Database 12.1.0.2 utPLSQL-cli 3.1.6.local
utPLSQL-java-api 3.1.6.411
Java-Version: 1.8.0_211
ojdbc7.jar

When i first executed the command:

../utPLSQL-cli/bin/utplsql run -d user/passwd@mydb:1521/mysid -source_path=sources -test_path=tests -f=ut_documentation_reporter -c -f=ut_coverage_sonar_reporter -o=coverage.xml -f=ut_sonar_test_reporter -o=test_results.xml --failure-exit-code=0

The command hanged for sometime with

##################### utPLSQL cli ######################
#                                                      #
#   utPLSQL-cli 3.1.6.local                            #
#   utPLSQL-java-api 3.1.6.411                         #
#   Java-Version: 1.8.0_211                            #
#   ORACLE_HOME: /u01/app/oracle/product/12.1.0/db_1   #
#   NLS_LANG: null                                     #
#                                                      #
#   Thanks for testing!                                #
#                                                      #
########################################################

HikariPool-1 - Driver does not support get/set network timeout for connections. (Recurso não suportado)
Use connection string jdbc:oracle:oci8:****/****@mydb:1521/mysid
Successfully connected to database. UtPLSQL core: v3.1.6.2735
Oracle-Version: 12.1.0.2.0
Running tests now.
--------------------------------------
TestRunner initialized
Running on utPLSQL v3.1.6.2735
Initializing reporters
Detected Oracle driver stuck during Statement initialization
WARNING: Caught Oracle stuck during creation of Runner-Statement. Retrying (1)

After a quite long wait time, and every time it retried the warning was reissued

##################### utPLSQL cli ######################
#                                                      #
#   utPLSQL-cli 3.1.6.local                            #
#   utPLSQL-java-api 3.1.6.411                         #
#   Java-Version: 1.8.0_211                            #
#   ORACLE_HOME: /u01/app/oracle/product/12.1.0/db_1   #
#   NLS_LANG: null                                     #
#                                                      #
#   Thanks for testing!                                #
#                                                      #
########################################################

HikariPool-2 - Driver does not support get/set network timeout for connections. (Recurso não suportado)
Use connection string jdbc:oracle:oci8:****/****@mydb:1521/mysid
Successfully connected to database. UtPLSQL core: v3.1.6.2735
Oracle-Version: 12.1.0.2.0
Running tests now.
--------------------------------------
TestRunner initialized
Running on utPLSQL v3.1.6.2735
Initializing reporters
Detected Oracle driver stuck during Statement initialization
WARNING: Caught Oracle stuck during creation of Runner-Statement. Retrying (2)
##################### utPLSQL cli ######################
#                                                      #
#   utPLSQL-cli 3.1.6.local                            #
#   utPLSQL-java-api 3.1.6.411                         #
#   Java-Version: 1.8.0_211                            #
#   ORACLE_HOME: /u01/app/oracle/product/12.1.0/db_1   #
#   NLS_LANG: null                                     #
#                                                      #
#   Thanks for testing!                                #
#                                                      #
########################################################

HikariPool-3 - Driver does not support get/set network timeout for connections. (Recurso não suportado)
Use connection string jdbc:oracle:oci8:****/****@mydb:1521/mysid
Successfully connected to database. UtPLSQL core: v3.1.6.2735
Oracle-Version: 12.1.0.2.0
Running tests now.
--------------------------------------
TestRunner initialized
Running on utPLSQL v3.1.6.2735
Initializing reporters
Detected Oracle driver stuck during Statement initialization
WARNING: Caught Oracle stuck during creation of Runner-Statement. Retrying (3)

after the 4th attempt the test runned smoothly:

##################### utPLSQL cli ######################
#                                                      #
#   utPLSQL-cli 3.1.6.local                            #
#   utPLSQL-java-api 3.1.6.411                         #
#   Java-Version: 1.8.0_211                            #
#   ORACLE_HOME: /u01/app/oracle/product/12.1.0/db_1   #
#   NLS_LANG: null                                     #
#                                                      #
#   Thanks for testing!                                #
#                                                      #
########################################################

HikariPool-4 - Driver does not support get/set network timeout for connections. (Recurso não suportado)
Use connection string jdbc:oracle:oci8:****/****@mydb:1521/mysid
Successfully connected to database. UtPLSQL core: v3.1.6.2735
Oracle-Version: 12.1.0.2.0
Running tests now.
--------------------------------------
TestRunner initialized
Running on utPLSQL v3.1.6.2735
Initializing reporters
Running tests
teste de teste
  resultado final [,046 sec] (FAILED - 1)


Failures:

  1) test_resultado_final
      Actual: 'TESTE NAO OK' (varchar2) was expected to equal: 'Teste OK' (varchar2)
      at "ESMPU.TEST_PKG_TESTE", line 4 ut.expect(pkg_teste.resultado_final).to_equal('Teste OK');


Finished in ,048334 seconds
1 tests, 1 failed, 0 errored, 0 disabled, 0 warning(s)

--------------------------------------
All tests done.

And now there's no more wait time in new executions.

I'm sending my current debug output, maybe it can help solving it.

and Hey! utPLSQL is awesome! utplsql-cli-log.txt

guicampos avatar Jun 05 '19 20:06 guicampos

Hey @DrErique @dmpe @guicampos could you try out the latest cli and tell me if you still experience the stuck problems? https://bintray.com/utplsql/utPLSQL-cli/download_file?file_path=utPLSQL-cli-develop-201906272140.zip

pesse avatar Jun 27 '19 21:06 pesse

Hello Samuel,

Thanks for contacting. I'll try some builds with that next week and tell you about the results.

Nice weekend !

Mit freundlichen Grüßen

Derik Wehrmann


Tel: -551 | Test- und Releasemanager | IT SCM

Von: Samuel Nitsche [email protected] Gesendet: Donnerstag, 27. Juni 2019 23:49 An: utPLSQL/utPLSQL-cli [email protected] Cc: Wehrmann, Derik [email protected]; Mention [email protected] Betreff: Re: [utPLSQL/utPLSQL-cli] Detected Oracle Driver stuck during Statement initialization (#145)

Hey @DrEriquehttps://github.com/DrErique @dmpehttps://github.com/dmpe @guicamposhttps://github.com/guicampos could you try out the latest cli and tell me if you still experience the stuck problems? https://bintray.com/utplsql/utPLSQL-cli/download_file?file_path=utPLSQL-cli-develop-201906272140.zip

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/utPLSQL/utPLSQL-cli/issues/145?email_source=notifications&email_token=AIPTY44HQ6ISYACF6PY3HCDP4UYTTA5CNFSM4HJLZYZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODYYPEEQ#issuecomment-506524178, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIPTY44MEWJIDLG6DYQN34LP4UYTTANCNFSM4HJLZYZA.

DrErique avatar Jun 28 '19 09:06 DrErique

############### utPLSQL cli ###############
#                                         #
#   utPLSQL-cli 3.1.7-SNAPSHOT.local      #
#   utPLSQL-java-api 3.1.7-SNAPSHOT.483   #
#   Java-Version: 1.8.0_212               #
#   ORACLE_HOME: null                     #
#   NLS_LANG: AMERICAN_AMERICA.UTF8       #
#                                         #
#   Thanks for testing!                   #
#                                         #
###########################################
Use connection string jdbc:oracle:thin:
Successfully connected to database. UtPLSQL core: v3.1.1.1868
Oracle-Version: 11.2.0.4.0
Running tests now.
--------------------------------------
TestRunner initialized
Running on utPLSQL v3.1.1.1868
Initializing reporters
Detected Oracle driver stuck during Statement initialization
Timeout while waiting for reporters to finish for 60 minutes

Thanks for the effort!!! It is better now but getting some errors. unfortunately cannot debug this today more extensively. Only later in the month.

dmpe avatar Jun 28 '19 14:06 dmpe

Is this the complete log of the problematic run?

Am Fr., 28. Juni 2019 um 16:07 Uhr schrieb John [email protected]:

Use connection string jdbc:oracle:thin: Successfully connected to database. UtPLSQL core: v3.1.1.1868 Oracle-Version: 11.2.0.4.0 Running tests now.

TestRunner initialized Running on utPLSQL v3.1.1.1868 Initializing reporters Detected Oracle driver stuck during Statement initialization Timeout while waiting for reporters to finish for 60 minutes

Thanks for the effort!!! It is better now but getting some errors. unfortunately cannot debug this today more extensively. Only later in the month.

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/utPLSQL/utPLSQL-cli/issues/145?email_source=notifications&email_token=AB7ZXYH4SQ4LMUCDDFFK6S3P4YLI5A5CNFSM4HJLZYZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY2FREA#issuecomment-506747024, or mute the thread https://github.com/notifications/unsubscribe-auth/AB7ZXYEY2KYYRGQIGUCLMB3P4YLI5ANCNFSM4HJLZYZA .

pesse avatar Jun 28 '19 14:06 pesse

Yes. I have just deleted login info

EDIT: I have used -source_path=. -test_path=. -f=ut_coverage_html_reporter -o=coverage.html -f=UT_DOCUMENTATION_REPORTER -s parameters but my issues may be related to ent. environment and the fact of very old core utplsql

dmpe avatar Jun 28 '19 14:06 dmpe

Thank you for trying out. I'm a bit lost on this, but will try to work my way around it. I might create a new, dedicated issue for that.

Am Fr., 28. Juni 2019 um 16:09 Uhr schrieb John [email protected]:

Yes. I have just deleted login info

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/utPLSQL/utPLSQL-cli/issues/145?email_source=notifications&email_token=AB7ZXYBFPPJYWX2O555HA3TP4YLQZA5CNFSM4HJLZYZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY2FWUI#issuecomment-506747729, or mute the thread https://github.com/notifications/unsubscribe-auth/AB7ZXYA7PAOHF5XTKJS3BNLP4YLQZANCNFSM4HJLZYZA .

pesse avatar Jun 28 '19 14:06 pesse

Hello Samuel,

Good news. The first couples of tests with the Bamboo Pipeline were successful. The client was able to hold the connection to the Database for a duration of 20 – 30 Minutes without getting stucked or other problems. I am very hopeful that we can establish firmly connections to the database with the client in the future.

For the first thank you for your work and the kindly conversation.

Mit freundlichen Grüßen

Derik Wehrmann


Tel: -551 | Test- und Releasemanager | IT SCM

Von: Samuel Nitsche [email protected] Gesendet: Freitag, 28. Juni 2019 16:12 An: utPLSQL/utPLSQL-cli [email protected] Cc: Wehrmann, Derik [email protected]; Mention [email protected] Betreff: Re: [utPLSQL/utPLSQL-cli] Detected Oracle Driver stuck during Statement initialization (#145)

Thank you for trying out. I'm a bit lost on this, but will try to work my way around it. I might create a new, dedicated issue for that.

Am Fr., 28. Juni 2019 um 16:09 Uhr schrieb John <[email protected]mailto:[email protected]>:

Yes. I have just deleted login info

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/utPLSQL/utPLSQL-cli/issues/145?email_source=notifications&email_token=AB7ZXYBFPPJYWX2O555HA3TP4YLQZA5CNFSM4HJLZYZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY2FWUI#issuecomment-506747729, or mute the thread https://github.com/notifications/unsubscribe-auth/AB7ZXYA7PAOHF5XTKJS3BNLP4YLQZANCNFSM4HJLZYZA .

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/utPLSQL/utPLSQL-cli/issues/145?email_source=notifications&email_token=AIPTY47PQ3K6PQVJ5E3KLHLP4YL27A5CNFSM4HJLZYZKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODY2F5SY#issuecomment-506748619, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AIPTY43UCNGDXCDN4VOJBWDP4YL27ANCNFSM4HJLZYZA.

DrErique avatar Jul 08 '19 15:07 DrErique

Thanks for reporting back, @DrErique I'll create a new, dedicated issue for the situations that still lead to a stuck scenario.

pesse avatar Jul 08 '19 18:07 pesse

Hi everyone,

Is there any news on this issue ?

I have come across the same problem "Detected Oracle driver stuck during Statement initialization" when trying to calculate the coverage:

2019-08-19 11:50:54 [main] INFO  org.utplsql.cli.RunAction - ############### utPLSQL cli ###############
#                                         #
#   utPLSQL-cli 3.1.7-SNAPSHOT.local      #
#   utPLSQL-java-api 3.1.7-SNAPSHOT.483   #
#   Java-Version: 1.8.0_201               #
#   ORACLE_HOME: null                     #
#   NLS_LANG: null                        #
#                                         #
#   Thanks for testing!                   #
#                                         #
###########################################
2019-08-19 11:50:54 [main] INFO  org.utplsql.cli.RunAction -
2019-08-19 11:50:55 [main] INFO  o.u.c.d.TestedDataSourceProvider - Use connection string jdbc:oracle:oci8:****/****@****:****:****
2019-08-19 11:50:57 [main] INFO  org.utplsql.cli.RunAction - Successfully connected to database. UtPLSQL core: v3.1.7.3085
2019-08-19 11:50:57 [main] INFO  org.utplsql.cli.RunAction - Oracle-Version: 12.2.0.1.0
2019-08-19 11:51:05 [main] DEBUG org.utplsql.api.reporter.Reporter - Database-reporter initialized, Type: UT_COVERAGE_HTML_REPORTER, ID: 90771DD22B780564E0530100007F1016
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - BaseDir: C:\work\repo\Main\db\utPLSQL-cli\bin
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - SourcePath: ../../test_proc
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - Getting FileMapperOptions - Files:
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - ..\..\test_proc\test_lib.pkb
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - ..\..\test_proc\test_lib.pks
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - BaseDir: C:\work\repo\Main\db\utPLSQL-cli\bin
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - SourcePath: ../../test_unit_testing
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - Getting FileMapperOptions - Files:
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - ..\..\test_unit_testing\test_test_lib.pkb
2019-08-19 11:51:05 [main] DEBUG org.utplsql.cli.RunAction - ..\..\test_unit_testing\test_test_lib.pks
2019-08-19 11:51:06 [pool-10-thread-1] INFO  org.utplsql.cli.RunTestRunnerTask - Running tests now.
2019-08-19 11:51:06 [pool-10-thread-1] INFO  org.utplsql.cli.RunTestRunnerTask - --------------------------------------
2019-08-19 11:51:06 [pool-10-thread-1] INFO  org.utplsql.api.TestRunner - TestRunner initialized
2019-08-19 11:51:06 [pool-10-thread-1] INFO  org.utplsql.api.TestRunner - Running on utPLSQL v3.1.7.3085
2019-08-19 11:51:06 [pool-10-thread-1] INFO  org.utplsql.api.TestRunner - Initializing reporters
2019-08-19 11:51:08 [pool-11-thread-1] DEBUG org.utplsql.api.FileMapper - Building fileMappingArray
2019-08-19 11:51:08 [pool-11-thread-1] DEBUG org.utplsql.api.FileMapper - Path: ..\..\test_proc\test_lib.pkb
2019-08-19 11:51:08 [pool-11-thread-1] DEBUG org.utplsql.api.FileMapper - Path: ..\..\test_proc\test_lib.pks
2019-08-19 11:51:08 [pool-10-thread-1] ERROR org.utplsql.api.TestRunner - Detected Oracle driver stuck during Statement initialization
2019-08-19 11:52:10 [main] WARN  org.utplsql.cli.RunAction - WARNING: Caught Oracle stuck during creation of Runner-Statement. Retrying (4)

If it's just the run of the tests then it works:

############### utPLSQL cli ###############
#                                         #
#   utPLSQL-cli 3.1.7-SNAPSHOT.local      #
#   utPLSQL-java-api 3.1.7-SNAPSHOT.483   #
#   Java-Version: 1.8.0_201               #
#   ORACLE_HOME: null                     #
#   NLS_LANG: null                        #
#                                         #
#   Thanks for testing!                   #
#                                         #
###########################################

Use connection string jdbc:oracle:oci8:****/****@****:****:****
Successfully connected to database. UtPLSQL core: v3.1.7.3096
Oracle-Version: 12.2.0.1.0
Running tests now.
--------------------------------------
TestRunner initialized
Running on utPLSQL v3.1.7.3096
Initializing reporters
Running tests
****
Running tests finished.

Finished in 11.875065 seconds
36 tests, 0 failed, 0 errored, 0 disabled, 0 warning(s)

--------------------------------------
All tests done.

Thanks & Regards, Natalia

kapbka avatar Aug 19 '19 11:08 kapbka

Hi All,

I have recently started doing POC on the utPLSQL for using it as a unit test frame work for PLSQL developments.

But I am encountering below issue when try executing the test from CLI.

Kindly help.

WARNING: Caught Oracle stuck during creation of Runner-Statement. Retrying (1)
############# utPLSQL cli ##############
#                                      #
#   utPLSQL-cli 3.1.8-SNAPSHOT.local   #
#   utPLSQL-java-api 3.1.8.546         #
#   Java-Version: 11.0.2               #
#   ORACLE_HOME: null                  #
#   NLS_LANG: null                     #
#                                      #
#   Thanks for testing!                #
#                                      #
########################################

Use connection string jdbc:oracle:thin:****/****@//XXXXX:1521/XXXXX
Successfully connected to database. UtPLSQL core: v3.1.9.3270
Oracle-Version: 12.1.0.2.0
Running tests now.
--------------------------------------
TestRunner initialized
Running on utPLSQL v3.1.9.3270
Initializing reporters
Detected Oracle driver stuck during Statement initialization
java.lang.NullPointerException
	at oracle.net.nt.TcpNTAdapter.abort(TcpNTAdapter.java:423)
	at oracle.net.ns.NSProtocol.abort(NSProtocol.java:532)
	at oracle.jdbc.driver.T4CConnection.doAbort(T4CConnection.java:5148)
	at oracle.jdbc.driver.PhysicalConnection.abort(PhysicalConnection.java:9353)
	at org.utplsql.cli.RunTestRunnerTask.call(RunTestRunnerTask.java:50)
	at org.utplsql.cli.RunTestRunnerTask.call(RunTestRunnerTask.java:24)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

vasanthsumanath avatar Dec 04 '19 15:12 vasanthsumanath

Hey @vasanthsumanath , can you please provide a full log as file, running the cli with the -d flag?

pesse avatar Dec 04 '19 15:12 pesse

Just to add, its working fine when connected to a different DB,

Below are the major difference between the 2 DB environments

Successfully executed in

Oracle-Version: 12.2.0.1.0 Platform - Docker image

Failed Executing from Oracle-Version: 12.1.0.2.0 Platform - CentOS 7 VM

############# utPLSQL cli ##############
#                                      #
#   utPLSQL-cli 3.1.8-SNAPSHOT.local   #
#   utPLSQL-java-api 3.1.8.546         #
#   Java-Version: 11.0.2               #
#   ORACLE_HOME: null                  #
#   NLS_LANG: null                     #
#                                      #
#   Thanks for testing!                #
#                                      #
########################################

Use connection string jdbc:oracle:thin:****/****@//localhost:1521/orclpdb1
Successfully connected to database. UtPLSQL core: v3.1.9.3270
Oracle-Version: 12.2.0.1.0
Running tests now.
--------------------------------------
TestRunner initialized
Running on utPLSQL v3.1.9.3270
Initializing reporters
Running tests
Between string function
Running tests finished.
  Returns substring from start position to end position [.056 sec]
  Returns substring when start position is zero [.003 sec]

Finished in .070032 seconds
2 tests, 0 failed, 0 errored, 0 disabled, 0 warning(s)

--------------------------------------
All tests done. 

vasanthsumanath avatar Dec 04 '19 15:12 vasanthsumanath

Hi @pesse,

Thanks for your quick response, yes will do it.

Thanks Vasanth.M

vasanthsumanath avatar Dec 04 '19 15:12 vasanthsumanath

Hey @vasanthsumanath , can you please provide a full log as file, running the cli with the -d flag?

Hi @pesse

I have attached log when executed in debug mode.

Below the command executed utplsql run ***/***@//***:1521/*** -d 1>>utplsql_cli.log 2>&1

utplsql_cli.log

vasanthsumanath avatar Dec 04 '19 15:12 vasanthsumanath

Any help/suggestion with the issue I am facing?

vasanthsumanath avatar Dec 11 '19 11:12 vasanthsumanath

These issues are among the hardest to investigate, because they are near impossible to reproduce on a different database. You also found a possible solution yourself already, for it seems to work on Oracle 12.2. This suggests some kind of Oracle DB bug which is even harder to work around.

My guess from the error message is that it's not really a "Stuck" problem as we have faced it in the past, but some general issue with the driver.

What you could do and that would probably help to investigate:

  • Make sure you have the latest DB patches installed
  • Try running ut.run() from within SQLcl (not SQLPlus)
  • Try out different OJDBC-Versions
  • Try out OJDBC-OCI (thick driver)

pesse avatar Dec 11 '19 12:12 pesse

Hi guys, I've started facing this issue too. I can say it's indeed intermittent and it's likely to happen more often (don't know why) when using the CLI in a pipeline. In my case here I am using GitHub Actions and there are times the test step fails because of this. If I simply hit "re-run jobs" I got cases that it worked on the first re-try and other it took more re-tries to work.

If any log is needed or further information that may help troubleshooting this, let me know.

rafael-trevisan avatar Jun 25 '20 18:06 rafael-trevisan

Which Oracle version are you on? Latest patchs applied?

Same is true for you as for Vasanthsumanath:

What you could do and that would probably help to investigate:

  • Make sure you have the latest DB patches installed
  • Try running ut.run() from within SQLcl (not SQLPlus)
  • Try out different OJDBC-Versions
  • Try out OJDBC-OCI (thick driver)

pesse avatar Jun 25 '20 20:06 pesse

I am using Oracle SE2 on AWS RDS. I am not sure about where it's in patches but RDS's settings are to upgrade even minor patches.

I am not installing any custom version of OJDBC. I am getting the CLI from

wget https://github.com/utPLSQL/utPLSQL-cli/releases/download/v3.1.8/utPLSQL-cli.zip

and I assume it contains the latest OJDBC, am I right?

If I want to try different version of OJDBC is it just replacing the existing one shipped with the CLI with whatever version I want to test?

rafael-trevisan avatar Jun 25 '20 20:06 rafael-trevisan