servicecomb-pack icon indicating copy to clipboard operation
servicecomb-pack copied to clipboard

Duplicated database record when save TxTimeoutEvent

Open wyzssw opened this issue 7 years ago • 9 comments

Alpha-Server Print Error Log when execute code

private void findTimeoutEvents() {
    eventRepository.findTimeoutEvents()
        .forEach(event -> {
          LOG.info("Found timeout event {}", event);
          timeoutRepository.save(txTimeoutOf(event));
        });
  }
2018-07-11 18:08:28.406  INFO 3700 --- [pool-2-thread-1] o.a.s.saga.alpha.core.EventScanner       : Found timeout event TxEvent{surrogateId=20, serviceName='booking', instanceId='booking-127.0.0.1', creationTime=Wed Jul 11 17:31:29 CST 2018, globalTxId='94fc1a1f-8dff-41bd-bb7b-82984456ed1c', localTxId='94fc1a1f-8dff-41bd-bb7b-82984456ed1c', parentTxId='null', type='SagaStartedEvent', compensationMethod='', expiryTime=Thu Jul 12 10:11:29 CST 2018, retryMethod='', retries=0}
[EL Warning]: 2018-07-11 18:08:31.446--UnitOfWork(1680426037)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.7.1.v20171221-bd47e8f): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '20' for key 'eventId'
Error Code: 1062
Call: INSERT INTO TxTimeout (EVENTID, EXPIRYTIME, GLOBALTXID, INSTANCEID, LOCALTXID, PARENTTXID, SERVICENAME, STATUS, TYPE, VERSION) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
	bind => [10 parameters bound]
Query: InsertObjectQuery(TxTimeout{eventId=20, serviceName='booking', instanceId='booking-127.0.0.1', globalTxId='94fc1a1f-8dff-41bd-bb7b-82984456ed1c', localTxId='94fc1a1f-8dff-41bd-bb7b-82984456ed1c', parentTxId='null', type='SagaStartedEvent', expiryTime=Thu Jul 12 10:11:29 CST 2018, status=NEW})
2018-07-11 18:08:31.452  WARN 3700 --- [pool-2-thread-1] o.a.s.s.a.s.SpringTxTimeoutRepository    : Failed to save some timeout TxTimeout{eventId=20, serviceName='booking', instanceId='booking-127.0.0.1', globalTxId='94fc1a1f-8dff-41bd-bb7b-82984456ed1c', localTxId='94fc1a1f-8dff-41bd-bb7b-82984456ed1c', parentTxId='null', type='SagaStartedEvent', expiryTime=Thu Jul 12 10:11:29 CST 2018, status=NEW}

wyzssw avatar Jul 11 '18 10:07 wyzssw

Can I know which version of Saga are you using? I just went through the code, maybe we need to find the distinct timeout events here.

WillemJiang avatar Jul 11 '18 14:07 WillemJiang

<parent>
   <artifactId>saga</artifactId>
   <groupId>org.apache.servicecomb.saga</groupId>
   <version>0.3.0-SNAPSHOT</version>
 </parent>
 <modelVersion>4.0.0</modelVersion>

 <artifactId>alpha</artifactId>
 <name>Saga::Alpha</name>

 <packaging>pom</packaging>
 <modules>
   <module>alpha-core</module>
   <module>alpha-server</module>
 </modules>

wyzssw avatar Jul 12 '18 06:07 wyzssw

How did you start the Alpha Server? We may hit the same issue of #158, we may resolve the issue by starting only one scanner.

WillemJiang avatar Jul 12 '18 06:07 WillemJiang

I start it in Intellij idea by debug AlphaApplication only one instance ,and only one eventScaner image with mysql database (schema-mysql.sql already imported)

wyzssw avatar Jul 12 '18 11:07 wyzssw

How did you start the Alpha Server? We may hit the same issue of #158, we may resolve the issue by starting only one scanner.

Several Alpha Servers were started as a cluster, how to start one scanner only? Any configurations? And how to handle if this server is dead? Thank you!

jiuang avatar Apr 28 '19 14:04 jiuang

Can you try the latest released pack 0.4.0 which has the server side cluster? There is only one event scanner started as a master in the cluster, if the master node is crashed, the other Alpha instance could be turned into a new master to scanne the events.

WillemJiang avatar Apr 29 '19 00:04 WillemJiang

Can you try the latest released pack 0.4.0 which has the server side cluster? There is only one event scanner started as a master in the cluster, if the master node is crashed, the other Alpha instance could be turned into a new master to scanne the events.

Okay, I will be try. Thanks!

jiuang avatar Apr 29 '19 02:04 jiuang

Caused by: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.7.1.v20171221-bd47e8f): org.eclipse.persistence.exceptions.DatabaseException Internal Exception: java.sql.SQLIntegrityConstraintViolationException: Duplicate entry 'c9ab7238-15db-4af8-be32-cf19e9854449-c9ab7238-15db-4af8-be32-cf1' for key 'tcc_global_tx_event_index' Error Code: 1062 Call: INSERT INTO tcc_global_tx_event (CREATIONTIME, GLOBALTXID, INSTANCEID, LASTMODIFIED, LOCALTXID, PARENTTXID, SERVICENAME, STATUS, TXTYPE) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?) bind => [9 parameters bound] Query: InsertObjectQuery(GlobalTxEvent{surrogateId=null, globalTxId='c9ab7238-15db-4af8-be32-cf19e9854449', localTxId='c9ab7238-15db-4af8-be32-cf19e9854449', parentTxId='', serviceName='ordering', instanceId='ordering-192.168.1.228', txType='END_TIMEOUT', status='Failed', creationTime=Thu Aug 22 17:08:37 CST 2019, lastModified=Thu Aug 22 17:08:37 CST 2019}) at org.eclipse.persistence.exceptions.DatabaseException.sqlException(DatabaseException.java:331) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeDirectNoSelect(DatabaseAccessor.java:905) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeNoSelect(DatabaseAccessor.java:967) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.basicExecuteCall(DatabaseAccessor.java:637) at org.eclipse.persistence.internal.databaseaccess.DatabaseAccessor.executeCall(DatabaseAccessor.java:564) at org.eclipse.persistence.internal.sessions.AbstractSession.basicExecuteCall(AbstractSession.java:2093) at org.eclipse.persistence.sessions.server.ClientSession.executeCall(ClientSession.java:309) at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:270) at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.executeCall(DatasourceCallQueryMechanism.java:256) at org.eclipse.persistence.internal.queries.DatasourceCallQueryMechanism.insertObject(DatasourceCallQueryMechanism.java:405) at org.eclipse.persistence.internal.queries.StatementQueryMechanism.insertObject(StatementQueryMechanism.java:165) at org.eclipse.persistence.internal.queries.StatementQueryMechanism.insertObject(StatementQueryMechanism.java:180) at org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.insertObjectForWrite(DatabaseQueryMechanism.java:502) at org.eclipse.persistence.queries.InsertObjectQuery.executeCommit(InsertObjectQuery.java:80) at org.eclipse.persistence.queries.InsertObjectQuery.executeCommitWithChangeSet(InsertObjectQuery.java:90) at org.eclipse.persistence.internal.queries.DatabaseQueryMechanism.executeWriteWithChangeSet(DatabaseQueryMechanism.java:314) at org.eclipse.persistence.queries.WriteObjectQuery.executeDatabaseQuery(WriteObjectQuery.java:58) at org.eclipse.persistence.queries.DatabaseQuery.execute(DatabaseQuery.java:911) at org.eclipse.persistence.queries.DatabaseQuery.executeInUnitOfWork(DatabaseQuery.java:810) at org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWorkObjectLevelModifyQuery(ObjectLevelModifyQuery.java:108) at org.eclipse.persistence.queries.ObjectLevelModifyQuery.executeInUnitOfWork(ObjectLevelModifyQuery.java:85) at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.internalExecuteQuery(UnitOfWorkImpl.java:2979) at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1892) at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1874) at org.eclipse.persistence.internal.sessions.AbstractSession.executeQuery(AbstractSession.java:1824) at org.eclipse.persistence.internal.sessions.CommitManager.commitNewObjectsForClassWithChangeSet(CommitManager.java:227) at org.eclipse.persistence.internal.sessions.CommitManager.commitAllObjectsForClassWithChangeSet(CommitManager.java:194) at org.eclipse.persistence.internal.sessions.CommitManager.commitAllObjectsWithChangeSet(CommitManager.java:139) at org.eclipse.persistence.internal.sessions.AbstractSession.writeAllObjectsWithChangeSet(AbstractSession.java:4384) at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabase(UnitOfWorkImpl.java:1491) at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitToDatabaseWithChangeSet(UnitOfWorkImpl.java:1581) at org.eclipse.persistence.internal.sessions.RepeatableWriteUnitOfWork.commitRootUnitOfWork(RepeatableWriteUnitOfWork.java:278) at org.eclipse.persistence.internal.sessions.UnitOfWorkImpl.commitAndResume(UnitOfWorkImpl.java:1218) at org.eclipse.persistence.internal.jpa.transaction.EntityTransactionImpl.commit(EntityTransactionImpl.java:134) ... 24 more

Fxdemon avatar Aug 22 '19 09:08 Fxdemon

It looks like there are two timeout message are sent to alpha. We introduce state machine of Saga in 0.5.0. And we just cut a release candidate, please feel free to try it.

WillemJiang avatar Aug 24 '19 01:08 WillemJiang