oracle
ORA-03896: Unable to load the sharding wallet successfully.
Hi Everyone,
I've successfully deployed Oracle 21c Sharded Database using standalone Docker Containers. Everything works fine the first time and while the environment is not rebooted. The moment when I restart some of the containers or the complete VM, I am unable to perform any sharded ddl, because the wallet location somehow gets lots.
SQL> insert into my_table values (1,'test'); * ERROR at line 1: ORA-03896: Unable to load the sharding wallet successfully. ORA-06512: at "SYS.DBMS_SESSION", line 155 ORA-06512: at "GSMADMIN_INTERNAL.DBMS_GSM_POOLADMIN", line 25013 ORA-03896: Unable to load the sharding wallet successfully. ORA-06512: at "SYS.DBMS_SESSION", line 141 ORA-06512: at "GSMADMIN_INTERNAL.DBMS_GSM_POOLADMIN", line 24910 ORA-06512: at "GSMADMIN_INTERNAL.DBMS_GSM_POOLADMIN", line 25031 ORA-06512: at line 1
Below is the information about the wallet's location on the CATALOG, SHARD1 and SHARD2 containers.
CATALOG
##################################################################################### SQL> select value from v$parameter where name='wallet_root';
VALUE
/opt/oracle/product/21c/dbhome_1/admin
SQL> select guid from v$pdbs where con_id = sys_context('userenv','con_id');
GUID
020CCE4FEE460AD6E0636614000A9A39
bash-4.2$ pwd /opt/oracle/admin/CATCDB/xdb_wallet bash-4.2$ ls -ltr total 8 -rw-------. 1 oracle oinstall 3835 Aug 4 15:11 ewallet.p12 -rw-------. 1 oracle oinstall 3880 Aug 4 15:11 cwallet.sso
SQL> select value from v$parameter where name='wallet_root';
SHARD1
#####################################################################################
SQL> select value from v$parameter where name='wallet_root';
VALUE
/opt/oracle/oradata/dbconfig/ORCL1CDB/admin
SQL> alter session set container=ORCL1PDB;
SQL> select guid from v$pdbs where con_id = sys_context('userenv','con_id');
GUID
020CEA753FD90AEEE0636714000AF57F
bash-4.2$ pwd /opt/oracle/oradata/dbconfig/ORCL1CDB/admin/020CEA753FD90AEEE0636714000AF57F/shard
bash-4.2$ ls -ltr total 4 -rw-------. 1 oracle oinstall 2877 Aug 4 15:27 cwallet.sso
bash-4.2$ pwd /opt/oracle/oradata/dbconfig/ORCL1CDB/admin/shard
bash-4.2$ ls -ltr total 4 -rw-------. 1 oracle oinstall 2661 Aug 3 22:01 cwallet.sso
SHARD2
#####################################################################################
SQL> select value from v$parameter where name='wallet_root';
VALUE
/opt/oracle/oradata/dbconfig/ORCL2CDB/admin
SQL> alter session set container=ORCL2PDB;
SQL> select guid from v$pdbs where con_id = sys_context('userenv','con_id');
GUID
020D07702D360AF0E0636814000A24E5
bash-4.2$ pwd /opt/oracle/oradata/dbconfig/ORCL2CDB/admin/020D07702D360AF0E0636814000A24E5/shard
bash-4.2$ ls -ltr total 4 -rw-------. 1 oracle oinstall 2877 Aug 4 15:27 cwallet.sso
bash-4.2$ pwd /opt/oracle/oradata/dbconfig/ORCL2CDB/admin/shard
bash-4.2$ ls -ltr total 4 -rw-------. 1 oracle oinstall 2661 Aug 3 22:09 cwallet.sso
Can you please take a look into this?
Thank you, Lazar
Hi Lazar,
I tried to reproduce this issue but could not in my test setup using 21c RDBMS and GSM Images. Also, I can see there are two wallets for Catalog Database:
[opc@vm-oracle-sharding deploy]$ docker exec -it catalog /bin/bash bash-4.2$ bash-4.2$ sqlplus sys as sysdba
SQL*Plus: Release 21.0.0.0.0 - Production on Tue Aug 15 20:48:45 2023 Version 21.3.0.0.0
Copyright (c) 1982, 2021, Oracle. All rights reserved.
Enter password:
Connected to: Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production Version 21.3.0.0.0
SQL> select value from v$parameter where name='wallet_root';
VALUE
/opt/oracle/product/21c/dbhome_1/admin
SQL> select guid,con_id from v$pdbs;
GUID CON_ID
026CB878DD8104E0E0636614000A1776 2 026CC6A6EE580AD1E0636614000A96D9 3
SQL> show pdbs
CON_ID CON_NAME OPEN MODE RESTRICTED
2 PDB$SEED READ ONLY NO
3 CAT1PDB READ WRITE NO
SQL> exit Disconnected from Oracle Database 21c Enterprise Edition Release 21.0.0.0.0 - Production Version 21.3.0.0.0 bash-4.2$ cd /opt/oracle/product/21c/dbhome_1/admin bash-4.2$ ls -lrt total 0 drwxr-x---. 2 oracle oinstall 25 Aug 8 16:23 shard drwxr-x---. 3 oracle oinstall 19 Aug 8 16:23 026CC6A6EE580AD1E0636614000A96D9 bash-4.2$ ls -rlt /opt/oracle/product/21c/dbhome_1/admin/shard/ total 4 -rw-------. 1 oracle oinstall 2549 Aug 8 16:23 cwallet.sso bash-4.2$ ls -rlt /opt/oracle/product/21c/dbhome_1/admin/026CC6A6EE580AD1E0636614000A96D9/ total 0 drwxr-x---. 2 oracle oinstall 25 Aug 8 16:23 shard bash-4.2$ ls -rlt /opt/oracle/product/21c/dbhome_1/admin/026CC6A6EE580AD1E0636614000A96D9/shard/ total 4 -rw-------. 1 oracle oinstall 2829 Aug 8 16:39 cwallet.sso bash-4.2$ bash-4.2$
I am also able to run DDL queries after restart of docker containers after system reboot - "docker start catalog gsm1 gsm2 shard1 shard2" and "docker exec -it gsm1 bash -c "/u01/app/oracle/product/21c/gsmhome_1/bin/sqlplus sharding_demo/sharding_demo@SHARDDIRECTOR1""
Is there a way we consistently reproduce it?
