dcrdata
dcrdata copied to clipboard
dcrpg: Remove duplicates for `votes`, `misses` and `treasury` table
Closes #1978
- Duplicates for
votesremoved successfully:
2024-01-03 21:40:00.003 [INF] PSQL: Block import elapsed: 0.30 minutes, 145 blocks (8.11 blocks/s)
2024-01-03 21:40:00.003 [INF] PSQL: Beginning SYNC STAGE 2 of 5 (duplicate row removal).
2024-01-03 21:40:00.003 [INF] PSQL: Dropping temporary index on addresses(tx_vin_vout_row_id).
2024-01-03 21:40:00.170 [INF] PSQL: Finding and removing duplicate table rows before indexing...
2024-01-03 21:40:00.170 [INF] PSQL: Finding and removing duplicate vins entries...
2024-01-03 21:47:00.622 [INF] PSQL: Removed 0 duplicate vins entries.
2024-01-03 21:47:00.623 [INF] PSQL: Finding and removing duplicate vouts entries...
2024-01-03 21:55:25.063 [INF] PSQL: Removed 0 duplicate vouts entries.
2024-01-03 21:55:25.064 [INF] PSQL: Finding and removing duplicate transactions entries...
2024-01-03 21:56:49.243 [INF] PSQL: Removed 0 duplicate transactions entries.
2024-01-03 21:56:49.244 [INF] PSQL: Finding and removing duplicate agendas entries...
2024-01-03 21:56:49.255 [INF] PSQL: Removed 0 duplicate agendas entries.
2024-01-03 21:56:49.255 [INF] PSQL: Finding and removing duplicate agenda_votes entries...
2024-01-03 21:57:08.451 [INF] PSQL: Removed 0 duplicate agenda_votes entries.
2024-01-03 21:57:08.451 [INF] PSQL: Finding and removing duplicate votes entries...
2024-01-03 21:57:35.071 [INF] PSQL: Removed 8 duplicate votes entries.
2024-01-03 21:57:35.161 [INF] PSQL: Beginning SYNC STAGE 3 of 5 (table indexing and analyzing).
- Duplicates for
missesremoved successfully:
2024-01-03 22:42:58.134 [INF] PSQL: Beginning SYNC STAGE 2 of 5 (duplicate row removal).
2024-01-03 22:42:58.134 [INF] PSQL: Dropping temporary index on addresses(tx_vin_vout_row_id).
2024-01-03 22:42:58.287 [INF] PSQL: Finding and removing duplicate table rows before indexing...
2024-01-03 22:42:58.287 [INF] PSQL: Finding and removing duplicate vins entries...
2024-01-03 22:53:28.812 [INF] PSQL: Removed 0 duplicate vins entries.
2024-01-03 22:53:28.812 [INF] PSQL: Finding and removing duplicate vouts entries...
2024-01-03 23:08:03.250 [INF] PSQL: Removed 0 duplicate vouts entries.
2024-01-03 23:08:03.251 [INF] PSQL: Finding and removing duplicate transactions entries...
2024-01-03 23:10:11.626 [INF] PSQL: Removed 0 duplicate transactions entries.
2024-01-03 23:10:11.627 [INF] PSQL: Finding and removing duplicate agendas entries...
2024-01-03 23:10:11.652 [INF] PSQL: Removed 0 duplicate agendas entries.
2024-01-03 23:10:11.652 [INF] PSQL: Finding and removing duplicate agenda_votes entries...
2024-01-03 23:10:40.645 [INF] PSQL: Removed 0 duplicate agenda_votes entries.
2024-01-03 23:10:40.646 [INF] PSQL: Finding and removing duplicate votes entries...
2024-01-03 23:11:17.805 [INF] PSQL: Removed 0 duplicate votes entries.
2024-01-03 23:11:17.806 [INF] PSQL: Finding and removing duplicate misses entries...
2024-01-03 23:11:18.786 [INF] PSQL: Removed 2 duplicate misses entries.
2024-01-03 23:11:18.786 [INF] PSQL: Beginning SYNC STAGE 3 of 5 (table indexing and analyzing).
- Duplicates for
treasuryremoved successfully:
2024-01-04 15:07:34.407 [INF] PSQL: Beginning SYNC STAGE 2 of 5 (duplicate row removal).
2024-01-04 15:07:34.407 [INF] PSQL: Dropping temporary index on addresses(tx_vin_vout_row_id).
2024-01-04 15:07:34.718 [INF] PSQL: Finding and removing duplicate table rows before indexing...
2024-01-04 15:07:34.719 [INF] PSQL: Finding and removing duplicate vins entries...
2024-01-04 15:13:54.914 [INF] PSQL: Removed 0 duplicate vins entries.
2024-01-04 15:13:54.915 [INF] PSQL: Finding and removing duplicate vouts entries...
2024-01-04 15:23:43.447 [INF] PSQL: Removed 0 duplicate vouts entries.
2024-01-04 15:23:43.448 [INF] PSQL: Finding and removing duplicate transactions entries...
2024-01-04 15:25:10.908 [INF] PSQL: Removed 0 duplicate transactions entries.
2024-01-04 15:25:10.909 [INF] PSQL: Finding and removing duplicate agendas entries...
2024-01-04 15:25:10.925 [INF] PSQL: Removed 0 duplicate agendas entries.
2024-01-04 15:25:10.925 [INF] PSQL: Finding and removing duplicate agenda_votes entries...
2024-01-04 15:25:29.573 [INF] PSQL: Removed 0 duplicate agenda_votes entries.
2024-01-04 15:25:29.573 [INF] PSQL: Finding and removing duplicate votes entries...
2024-01-04 15:25:52.580 [INF] PSQL: Removed 0 duplicate votes entries.
2024-01-04 15:25:52.580 [INF] PSQL: Finding and removing duplicate misses entries...
2024-01-04 15:25:53.819 [INF] PSQL: Removed 0 duplicate misses entries.
2024-01-04 15:25:53.819 [INF] PSQL: Finding and removing duplicate treasury entries...
2024-01-04 15:25:56.690 [INF] PSQL: Removed 2 duplicate treasury entries.
2024-01-04 15:25:56.690 [INF] PSQL: Beginning SYNC STAGE 3 of 5 (table indexing and analyzing).
Turns out we also need to remove duplicates for uix_ticket_hashes_index.
2024-08-23 01:13:57.732 [INF] PSQL: Indexing treasury on tx hash...
2024-08-23 01:13:59.190 [INF] PSQL: Indexing treasury on block height...
2024-08-23 01:13:59.465 [INF] PSQL: Indexing swaps on spend height...
2024-08-23 01:13:59.963 [INF] PSQL: Indexing tickets table on ticket hash...
2024-08-23 01:14:06.736 [ERR] DATD: dcrpg.SyncChainDB failed at height 898324.
2024-08-23 01:14:06.736 [INF] DATD: Shutdown requested. Shutting down...
2024-08-23 01:14:06.736 [DBG] JAPI: Got quit signal. Exiting block connected handler for STATUS monitor.
2024-08-23 01:14:06.737 [INF] DATD: Gracefully shutting down web server...
2024-08-23 01:14:06.737 [DBG] DATD: Block prefetcher hits = 92, misses = 2.
2024-08-23 01:14:06.738 [INF] PUBS: Stopping websocket hub.
2024-08-23 01:14:06.739 [DBG] PUBS: Unregistered and killed 0 clients.
2024-08-23 01:14:06.739 [INF] EXPR: Stopping websocket hub.
2024-08-23 01:14:06.738 [WRN] EXPR: ExchangeBot has quit.
2024-08-23 01:14:06.739 [ERR] XBOT: dcrdex websocket error: read tcp 192.168.1.243:60761->45.77.164.152:7232: use of closed network connection
2024-08-23 01:14:06.901 [DBG] SKDB: badger: Got compaction priority: {level:0 score:1.73 dropPrefixes:[]}
2024-08-23 01:14:07.909 [DBG] SKDB: badger: LOG Compact. Added 898327 keys. Skipped 1 keys. Iteration took: 1.00608323s
2024-08-23 01:14:08.405 [DBG] SKDB: badger: Discard stats: map[]
2024-08-23 01:14:08.415 [INF] SKDB: badger: LOG Compact 0->1, del 2 tables, add 1 tables, took 1.513239135s
2024-08-23 01:14:08.415 [DBG] SKDB: badger: Compaction for level: 0 DONE
2024-08-23 01:14:08.508 [INF] DATD: Closing connection to dcrd.
2024-08-23 01:14:08.509 [INF] DATD: Bye!
2024-08-23 01:14:08.760 [ERR] DATD: IndexTicketsTable failed: pq: could not create unique index "uix_ticket_hashes_index"