celestia-node icon indicating copy to clipboard operation
celestia-node copied to clipboard

Bridge stops catching up in mocha testnet

Open shivlim opened this issue 1 year ago • 8 comments

Celestia Node version

Semantic version: v0.13.1 Commit: 32bc7a973b2f0b730c8ec1c2f7d6be5e5484e85b Build Date: Wed Mar 6 22:48:30 UTC 2024 System version: amd64/linux Golang version: go1.22.0

OS

AME="Ubuntu" VERSION="21.04 (Hirsute Hippo)" ID=ubuntu ID_LIKE=debian PRETTY_NAME="Ubuntu 21.04" VERSION_ID="21.04" HOME_URL="https://www.ubuntu.com/" SUPPORT_URL="https://help.ubuntu.com/" BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" VERSION_CODENAME=hirsute UBUNTU_CODENAME=hirsute

Install tools

No response

Others

No response

Steps to reproduce it

Bridge stops catchup and works only after restart.please look at logs. Stops working at 2024-03-21T10:15:07.215Z at height 1430198. (look at logs). Works after restart.

Expected result

Bridge should catchup

Actual result

stops catchup. process is still alive. works after restart

Relevant log output

Mar 21 10:14:08  celestia[30304]: 2024-03-21T10:14:08.813Z        INFO        header/store        store/store.go:375        new head        {"height": 1430193, "hash": "BF689EA2A3AAF5151D134EFDA916937E93789C77BA5AC8A9AE2D445E01634201"}
Mar 21 10:14:09  celestia[30304]: 2024-03-21T10:14:09.321Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27017567 miss: 704536390 keys-added: 53125978 keys-updated: 140218 keys-evicted: 53125966 cost-added: 556418852339327 cost-evicted: 556418597940642 sets-dropped: 1 sets-rejected: 454469197 gets-dropped: 27777216 gets-kept: 702379840 gets-total: 731553957 hit-ratio: 0.04
Mar 21 10:14:09  celestia[30304]: 2024-03-21T10:14:09.321Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:14:09  celestia[30304]:  -- Histogram:
Mar 21 10:14:09  celestia[30304]: Min value: 0
Mar 21 10:14:09  celestia[30304]: Max value: 30334
Mar 21 10:14:09  celestia[30304]: Count: 53118738
Mar 21 10:14:09  celestia[30304]: 50p: 2.00
Mar 21 10:14:09  celestia[30304]: 75p: 2.00
Mar 21 10:14:09  celestia[30304]: 90p: 2.00
Mar 21 10:14:09  celestia[30304]: [0, 2) 52020237 97.93% 97.93%
Mar 21 10:14:09  celestia[30304]: [2, 4) 565248 1.06% 99.00%
Mar 21 10:14:09  celestia[30304]: [4, 8) 274228 0.52% 99.51%
Mar 21 10:14:09  celestia[30304]: [8, 16) 149483 0.28% 99.79%
Mar 21 10:14:09  celestia[30304]: [16, 32) 52305 0.10% 99.89%
Mar 21 10:14:09  celestia[30304]: [32, 64) 30931 0.06% 99.95%
Mar 21 10:14:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:14:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:14:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:14:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:14:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:14:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:14:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:14:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:14:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:14:09  celestia[30304]:  --
Mar 21 10:14:20  celestia[30304]: 2024-03-21T10:14:20.549Z        INFO        header/store        store/store.go:375        new head        {"height": 1430194, "hash": "A80FB5D437A4FB1BC8EA6E8C64D130A7E0A7142D6E235BDDB937A1B27F48B3C2"}
Mar 21 10:14:32  celestia[30304]: 2024-03-21T10:14:32.195Z        INFO        header/store        store/store.go:375        new head        {"height": 1430195, "hash": "90FA826FE31FFE603D23C583ED65AC201C755F719F2E763E5B01D7388ECA55A2"}
Mar 21 10:14:33  celestia[30304]: 2024-03-21T10:14:33.069Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: stream reset
Mar 21 10:14:42  celestia[30304]: 2024-03-21T10:14:42.980Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:14:43  celestia[30304]: 2024-03-21T10:14:43.850Z        INFO        header/store        store/store.go:375        new head        {"height": 1430196, "hash": "F97F0C8ECBE28536147A065FD2753C8EA5C93CAF78904BCBF5ECF400446C5BC4"}
Mar 21 10:14:55  celestia[30304]: 2024-03-21T10:14:55.496Z        INFO        header/store        store/store.go:375        new head        {"height": 1430197, "hash": "EA43B4A649E8E5BD391BE2EC7C9A917B690BAB12C50FFE3DC786B02954452CAB"}
Mar 21 10:15:03  celestia[30304]: 2024-03-21T10:15:03.283Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: timeout: no recent network activity
Mar 21 10:15:07  celestia[30304]: 2024-03-21T10:15:07.215Z        INFO        header/store        store/store.go:375        new head        {"height": 1430198, "hash": "F3DD6B105274082043D193D6EAF6BD715D40356BA73ACB57591C17AB3A6524C7"}
Mar 21 10:15:09  celestia[30304]: 2024-03-21T10:15:09.322Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27020331 miss: 704584360 keys-added: 53130998 keys-updated: 140238 keys-evicted: 53130980 cost-added: 556472826693398 cost-evicted: 556472561761175 sets-dropped: 1 sets-rejected: 454512127 gets-dropped: 27777216 gets-kept: 702430656 gets-total: 731604691 hit-ratio: 0.04
Mar 21 10:15:09  celestia[30304]: 2024-03-21T10:15:09.322Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:15:09  celestia[30304]:  -- Histogram:
Mar 21 10:15:09  celestia[30304]: Min value: 0
Mar 21 10:15:09  celestia[30304]: Max value: 30334
Mar 21 10:15:09  celestia[30304]: Count: 53123752
Mar 21 10:15:09  celestia[30304]: 50p: 2.00
Mar 21 10:15:09  celestia[30304]: 75p: 2.00
Mar 21 10:15:09  celestia[30304]: 90p: 2.00
Mar 21 10:15:09  celestia[30304]: [0, 2) 52025088 97.93% 97.93%
Mar 21 10:15:09  celestia[30304]: [2, 4) 565361 1.06% 99.00%
Mar 21 10:15:09  celestia[30304]: [4, 8) 274267 0.52% 99.51%
Mar 21 10:15:09  celestia[30304]: [8, 16) 149493 0.28% 99.79%
Mar 21 10:15:09  celestia[30304]: [16, 32) 52306 0.10% 99.89%
Mar 21 10:15:09  celestia[30304]: [32, 64) 30931 0.06% 99.95%
Mar 21 10:15:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:15:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:15:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:15:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:15:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:15:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:15:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:15:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:15:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:15:09  celestia[30304]:  --
Mar 21 10:15:17  celestia[30304]: 2024-03-21T10:15:17.479Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:15:42  celestia[30304]: 2024-03-21T10:15:42.323Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWR6SHsXPkkvhCRn6vp1RqSefgaT1X1nMNvrVjU2o3GoYy", "hash": "FDC18FD4FD4B6E34AC49E1ED78AAB6849C63DBEC360DE4F838C3518C25B191A6"}
Mar 21 10:15:52  celestia[30304]: 2024-03-21T10:15:52.980Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:15:57  celestia[30304]: 2024-03-21T10:15:57.215Z        INFO        core        core/listener.go:196        listener: listening stopped
Mar 21 10:15:57  celestia[30304]: 2024-03-21T10:15:57.217Z        WARN        core        core/listener.go:129        listener: subscriber error, resubscribing...        {"err": "underlying subscription is stuck"}
Mar 21 10:16:05  celestia[30304]: 2024-03-21T10:16:05.792Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWNEDvy2uYwBrtic5BzfDioKHDdSoTC72jwzHewNoVv1B3", "hash": "177B53A10377E735C2AAA3B019BEEAE2333EE428CF6C78FD58578C0D95F72A05"}
Mar 21 10:16:06  celestia[30304]: 2024-03-21T10:16:06.068Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "177B53A10377E735C2AAA3B019BEEAE2333EE428CF6C78FD58578C0D95F72A05"}
Mar 21 10:16:09  celestia[30304]: 2024-03-21T10:16:09.321Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27022616 miss: 704626427 keys-added: 53135486 keys-updated: 140255 keys-evicted: 53135469 cost-added: 556520343122660 cost-evicted: 556520095954462 sets-dropped: 1 sets-rejected: 454549689 gets-dropped: 27777216 gets-kept: 702474944 gets-total: 731649043 hit-ratio: 0.04
Mar 21 10:16:09  celestia[30304]: 2024-03-21T10:16:09.321Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:16:09  celestia[30304]:  -- Histogram:
Mar 21 10:16:09  celestia[30304]: Min value: 0
Mar 21 10:16:09  celestia[30304]: Max value: 30334
Mar 21 10:16:09  celestia[30304]: Count: 53128241
Mar 21 10:16:09  celestia[30304]: 50p: 2.00
Mar 21 10:16:09  celestia[30304]: 75p: 2.00
Mar 21 10:16:09  celestia[30304]: 90p: 2.00
Mar 21 10:16:09  celestia[30304]: [0, 2) 52029420 97.93% 97.93%
Mar 21 10:16:09  celestia[30304]: [2, 4) 565469 1.06% 99.00%
Mar 21 10:16:09  celestia[30304]: [4, 8) 274307 0.52% 99.51%
Mar 21 10:16:09  celestia[30304]: [8, 16) 149502 0.28% 99.79%
Mar 21 10:16:09  celestia[30304]: [16, 32) 52306 0.10% 99.89%
Mar 21 10:16:09  celestia[30304]: [32, 64) 30931 0.06% 99.95%
Mar 21 10:16:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:16:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:16:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:16:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:16:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:16:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:16:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:16:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:16:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:16:09  celestia[30304]:  --
Mar 21 10:16:17  celestia[30304]: 2024-03-21T10:16:17.710Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "C085F6CD2DA4A0418C53924163D4AB502ACDB533FAB323AF1FDACA3422FC74B1"}
Mar 21 10:16:29  celestia[30304]: 2024-03-21T10:16:29.158Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWR6SHsXPkkvhCRn6vp1RqSefgaT1X1nMNvrVjU2o3GoYy", "hash": "8A4A7FBE31B6C848911E26EB45A44657F6AD3A315664E472706BB16FFDCFC4B5"}
Mar 21 10:17:09  celestia[30304]: 2024-03-21T10:17:09.321Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27025384 miss: 704672914 keys-added: 53140383 keys-updated: 140275 keys-evicted: 53140365 cost-added: 556572170199125 cost-evicted: 556571914817607 sets-dropped: 1 sets-rejected: 454591259 gets-dropped: 27777216 gets-kept: 702524096 gets-total: 731698298 hit-ratio: 0.04
Mar 21 10:17:09  celestia[30304]: 2024-03-21T10:17:09.321Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:17:09  celestia[30304]:  -- Histogram:
Mar 21 10:17:09  celestia[30304]: Min value: 0
Mar 21 10:17:09  celestia[30304]: Max value: 30334
Mar 21 10:17:09  celestia[30304]: Count: 53133137
Mar 21 10:17:09  celestia[30304]: 50p: 2.00
Mar 21 10:17:09  celestia[30304]: 75p: 2.00
Mar 21 10:17:09  celestia[30304]: 90p: 2.00
Mar 21 10:17:09  celestia[30304]: [0, 2) 52034140 97.93% 97.93%
Mar 21 10:17:09  celestia[30304]: [2, 4) 565593 1.06% 99.00%
Mar 21 10:17:09  celestia[30304]: [4, 8) 274351 0.52% 99.51%
Mar 21 10:17:09  celestia[30304]: [8, 16) 149510 0.28% 99.79%
Mar 21 10:17:09  celestia[30304]: [16, 32) 52306 0.10% 99.89%
Mar 21 10:17:09  celestia[30304]: [32, 64) 30931 0.06% 99.95%
Mar 21 10:17:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:17:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:17:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:17:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:17:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:17:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:17:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:17:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:17:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:17:09  celestia[30304]:  --
Mar 21 10:17:12  celestia[30304]: 2024-03-21T10:17:12.980Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:17:17  celestia[30304]: 2024-03-21T10:17:17.479Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:18:02  celestia[30304]: 2024-03-21T10:18:02.901Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "6CB24F2239FDE78C10EDD7E8277098DD657CC041EB4231325B31AD365D1B4AA2"}
Mar 21 10:18:09  celestia[30304]: 2024-03-21T10:18:09.322Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27028201 miss: 704720156 keys-added: 53144154 keys-updated: 140283 keys-evicted: 53144128 cost-added: 556609965490506 cost-evicted: 556609700888586 sets-dropped: 1 sets-rejected: 454634722 gets-dropped: 27777216 gets-kept: 702574336 gets-total: 731748357 hit-ratio: 0.04
Mar 21 10:18:09  celestia[30304]: 2024-03-21T10:18:09.322Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:18:09  celestia[30304]:  -- Histogram:
Mar 21 10:18:09  celestia[30304]: Min value: 0
Mar 21 10:18:09  celestia[30304]: Max value: 30334
Mar 21 10:18:09  celestia[30304]: Count: 53136900
Mar 21 10:18:09  celestia[30304]: 50p: 2.00
Mar 21 10:18:09  celestia[30304]: 75p: 2.00
Mar 21 10:18:09  celestia[30304]: 90p: 2.00
Mar 21 10:18:09  celestia[30304]: [0, 2) 52037797 97.93% 97.93%
Mar 21 10:18:09  celestia[30304]: [2, 4) 565656 1.06% 99.00%
Mar 21 10:18:09  celestia[30304]: [4, 8) 274375 0.52% 99.51%
Mar 21 10:18:09  celestia[30304]: [8, 16) 149520 0.28% 99.79%
Mar 21 10:18:09  celestia[30304]: [16, 32) 52314 0.10% 99.89%
Mar 21 10:18:09  celestia[30304]: [32, 64) 30932 0.06% 99.95%
Mar 21 10:18:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:18:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:18:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:18:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:18:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:18:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:18:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:18:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:18:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:18:09  celestia[30304]:  --
Mar 21 10:18:26  celestia[30304]: 2024-03-21T10:18:26.434Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "76206058E5625EC258E7C2142413A95FFE7852223FC98B4805779B5050307D63"}
Mar 21 10:18:37  celestia[30304]: 2024-03-21T10:18:37.973Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWCUHPLqQXZzpTx1x3TAsdn3vYmTNDhzg66yG8hqoxGGN8", "hash": "D755958B07FA5DE7F96B7D605AFC1C034FDC42B41EDEDC419889073883416E5E"}
Mar 21 10:18:42  celestia[30304]: 2024-03-21T10:18:42.981Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:18:49  celestia[30304]: 2024-03-21T10:18:49.634Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWCUHPLqQXZzpTx1x3TAsdn3vYmTNDhzg66yG8hqoxGGN8", "hash": "45FF43603EF78D4D6BDAEAED594FDAF9B783DBFEB29AA350D8DC09713F740702"}
Mar 21 10:19:09  celestia[30304]: 2024-03-21T10:19:09.322Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27030661 miss: 704769554 keys-added: 53148066 keys-updated: 140299 keys-evicted: 53148045 cost-added: 556644523301663 cost-evicted: 556644258873981 sets-dropped: 1 sets-rejected: 454680192 gets-dropped: 27777216 gets-kept: 702625984 gets-total: 731800215 hit-ratio: 0.04
Mar 21 10:19:09  celestia[30304]: 2024-03-21T10:19:09.322Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:19:09  celestia[30304]:  -- Histogram:
Mar 21 10:19:09  celestia[30304]: Min value: 0
Mar 21 10:19:09  celestia[30304]: Max value: 30334
Mar 21 10:19:09  celestia[30304]: Count: 53140817
Mar 21 10:19:09  celestia[30304]: 50p: 2.00
Mar 21 10:19:09  celestia[30304]: 75p: 2.00
Mar 21 10:19:09  celestia[30304]: 90p: 2.00
Mar 21 10:19:09  celestia[30304]: [0, 2) 52041539 97.93% 97.93%
Mar 21 10:19:09  celestia[30304]: [2, 4) 565735 1.06% 99.00%
Mar 21 10:19:09  celestia[30304]: [4, 8) 274430 0.52% 99.51%
Mar 21 10:19:09  celestia[30304]: [8, 16) 149543 0.28% 99.79%
Mar 21 10:19:09  celestia[30304]: [16, 32) 52325 0.10% 99.89%
Mar 21 10:19:09  celestia[30304]: [32, 64) 30939 0.06% 99.95%
Mar 21 10:19:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:19:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:19:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:19:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:19:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:19:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:19:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:19:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:19:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:19:09  celestia[30304]:  --
Mar 21 10:19:17  celestia[30304]: 2024-03-21T10:19:17.479Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:19:23  celestia[30304]: 2024-03-21T10:19:23.069Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: stream reset
Mar 21 10:19:35  celestia[30304]: 2024-03-21T10:19:35.941Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWCUHPLqQXZzpTx1x3TAsdn3vYmTNDhzg66yG8hqoxGGN8", "hash": "15A626F2C348DA4B42721749A7AB58D577438B497C26E1AE424A06F21829720E"}
Mar 21 10:19:36  celestia[30304]: 2024-03-21T10:19:36.679Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "15A626F2C348DA4B42721749A7AB58D577438B497C26E1AE424A06F21829720E"}
Mar 21 10:20:06  celestia[30304]: 2024-03-21T10:20:06.161Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "42684587E6F44EAB3B7C509FD17A42E1BEC2488E02FEAF7CCDCE331FEBC195E5"}
Mar 21 10:20:09  celestia[30304]: 2024-03-21T10:20:09.321Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27033685 miss: 704823369 keys-added: 53153225 keys-updated: 140318 keys-evicted: 53153206 cost-added: 556695354868759 cost-evicted: 556695087328444 sets-dropped: 1 sets-rejected: 454728829 gets-dropped: 27777280 gets-kept: 702682752 gets-total: 731857054 hit-ratio: 0.04
Mar 21 10:20:09  celestia[30304]: 2024-03-21T10:20:09.322Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:20:09  celestia[30304]:  -- Histogram:
Mar 21 10:20:09  celestia[30304]: Min value: 0
Mar 21 10:20:09  celestia[30304]: Max value: 30334
Mar 21 10:20:09  celestia[30304]: Count: 53145978
Mar 21 10:20:09  celestia[30304]: 50p: 2.00
Mar 21 10:20:09  celestia[30304]: 75p: 2.00
Mar 21 10:20:09  celestia[30304]: 90p: 2.00
Mar 21 10:20:09  celestia[30304]: [0, 2) 52046534 97.93% 97.93%
Mar 21 10:20:09  celestia[30304]: [2, 4) 565836 1.06% 99.00%
Mar 21 10:20:09  celestia[30304]: [4, 8) 274477 0.52% 99.51%
Mar 21 10:20:09  celestia[30304]: [8, 16) 149560 0.28% 99.79%
Mar 21 10:20:09  celestia[30304]: [16, 32) 52326 0.10% 99.89%
Mar 21 10:20:09  celestia[30304]: [32, 64) 30939 0.06% 99.95%
Mar 21 10:20:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:20:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:20:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:20:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:20:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:20:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:20:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:20:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:20:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:20:09  celestia[30304]:  --
Mar 21 10:20:12  celestia[30304]: 2024-03-21T10:20:12.979Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:20:33  celestia[30304]: 2024-03-21T10:20:33.069Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: stream reset
Mar 21 10:20:35  celestia[30304]: 2024-03-21T10:20:35.885Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: timeout: no recent network activity
Mar 21 10:21:09  celestia[30304]: 2024-03-21T10:21:09.322Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27036279 miss: 704869656 keys-added: 53158082 keys-updated: 140328 keys-evicted: 53158063 cost-added: 556744677379903 cost-evicted: 556744419502665 sets-dropped: 1 sets-rejected: 454770249 gets-dropped: 27777280 gets-kept: 702731520 gets-total: 731905935 hit-ratio: 0.04
Mar 21 10:21:09  celestia[30304]: 2024-03-21T10:21:09.322Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:21:09  celestia[30304]:  -- Histogram:
Mar 21 10:21:09  celestia[30304]: Min value: 0
Mar 21 10:21:09  celestia[30304]: Max value: 30334
Mar 21 10:21:09  celestia[30304]: Count: 53150835
Mar 21 10:21:09  celestia[30304]: 50p: 2.00
Mar 21 10:21:09  celestia[30304]: 75p: 2.00
Mar 21 10:21:09  celestia[30304]: 90p: 2.00
Mar 21 10:21:09  celestia[30304]: [0, 2) 52051218 97.93% 97.93%
Mar 21 10:21:09  celestia[30304]: [2, 4) 565939 1.06% 99.00%
Mar 21 10:21:09  celestia[30304]: [4, 8) 274531 0.52% 99.51%
Mar 21 10:21:09  celestia[30304]: [8, 16) 149575 0.28% 99.79%
Mar 21 10:21:09  celestia[30304]: [16, 32) 52327 0.10% 99.89%
Mar 21 10:21:09  celestia[30304]: [32, 64) 30939 0.06% 99.95%
Mar 21 10:21:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:21:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:21:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:21:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:21:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:21:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:21:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:21:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:21:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:21:09  celestia[30304]:  --
Mar 21 10:21:17  celestia[30304]: 2024-03-21T10:21:17.479Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:21:39  celestia[30304]: 2024-03-21T10:21:39.697Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "9A2A159322084DA28E82401328E528486F9B8AB1F6268C4289C8B1013D284FD5"}
Mar 21 10:21:42  celestia[30304]: 2024-03-21T10:21:42.980Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:22:02  celestia[30304]: 2024-03-21T10:22:02.955Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWR6SHsXPkkvhCRn6vp1RqSefgaT1X1nMNvrVjU2o3GoYy", "hash": "6C18FCF49C9FB5CBDE45D502F967BD92954447580E428834FECF8DFD478ABBE2"}
Mar 21 10:22:09  celestia[30304]: 2024-03-21T10:22:09.322Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27039078 miss: 704919208 keys-added: 53163369 keys-updated: 140348 keys-evicted: 53163354 cost-added: 556802463212882 cost-evicted: 556802198805417 sets-dropped: 1 sets-rejected: 454814494 gets-dropped: 27777280 gets-kept: 702783872 gets-total: 731958286 hit-ratio: 0.04
Mar 21 10:22:09  celestia[30304]: 2024-03-21T10:22:09.322Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:22:09  celestia[30304]:  -- Histogram:
Mar 21 10:22:09  celestia[30304]: Min value: 0
Mar 21 10:22:09  celestia[30304]: Max value: 30334
Mar 21 10:22:09  celestia[30304]: Count: 53156126
Mar 21 10:22:09  celestia[30304]: 50p: 2.00
Mar 21 10:22:09  celestia[30304]: 75p: 2.00
Mar 21 10:22:09  celestia[30304]: 90p: 2.00
Mar 21 10:22:09  celestia[30304]: [0, 2) 52056340 97.93% 97.93%
Mar 21 10:22:09  celestia[30304]: [2, 4) 566039 1.06% 99.00%
Mar 21 10:22:09  celestia[30304]: [4, 8) 274582 0.52% 99.51%
Mar 21 10:22:09  celestia[30304]: [8, 16) 149592 0.28% 99.79%
Mar 21 10:22:09  celestia[30304]: [16, 32) 52328 0.10% 99.89%
Mar 21 10:22:09  celestia[30304]: [32, 64) 30939 0.06% 99.95%
Mar 21 10:22:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:22:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:22:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:22:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:22:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:22:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:22:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:22:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:22:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:22:09  celestia[30304]:  --
Mar 21 10:22:23  celestia[30304]: 2024-03-21T10:22:23.070Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: stream reset
Mar 21 10:22:38  celestia[30304]: 2024-03-21T10:22:38.468Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "2488BF6777E8905380BB41D50631FE8F18909DAC36949EADE333C0B68E4728FD"}
Mar 21 10:22:52  celestia[30304]: 2024-03-21T10:22:52.980Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:23:01  celestia[30304]: 2024-03-21T10:23:01.852Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWCUHPLqQXZzpTx1x3TAsdn3vYmTNDhzg66yG8hqoxGGN8", "hash": "0E2DD2B180042425ED0C7361F79A3536836C458A06215C2B3D1A51BBF4DD9D96"}
Mar 21 10:23:01  celestia[30304]: 2024-03-21T10:23:01.974Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "0E2DD2B180042425ED0C7361F79A3536836C458A06215C2B3D1A51BBF4DD9D96"}
Mar 21 10:23:09  celestia[30304]: 2024-03-21T10:23:09.321Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27041766 miss: 704969220 keys-added: 53168376 keys-updated: 140357 keys-evicted: 53168357 cost-added: 556853157700772 cost-evicted: 556852889307885 sets-dropped: 1 sets-rejected: 454859490 gets-dropped: 27777280 gets-kept: 702836928 gets-total: 732010986 hit-ratio: 0.04
Mar 21 10:23:09  celestia[30304]: 2024-03-21T10:23:09.321Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:23:09  celestia[30304]:  -- Histogram:
Mar 21 10:23:09  celestia[30304]: Min value: 0
Mar 21 10:23:09  celestia[30304]: Max value: 30334
Mar 21 10:23:09  celestia[30304]: Count: 53161129
Mar 21 10:23:09  celestia[30304]: 50p: 2.00
Mar 21 10:23:09  celestia[30304]: 75p: 2.00
Mar 21 10:23:09  celestia[30304]: 90p: 2.00
Mar 21 10:23:09  celestia[30304]: [0, 2) 52061175 97.93% 97.93%
Mar 21 10:23:09  celestia[30304]: [2, 4) 566155 1.06% 99.00%
Mar 21 10:23:09  celestia[30304]: [4, 8) 274630 0.52% 99.51%
Mar 21 10:23:09  celestia[30304]: [8, 16) 149596 0.28% 99.79%
Mar 21 10:23:09  celestia[30304]: [16, 32) 52328 0.10% 99.89%
Mar 21 10:23:09  celestia[30304]: [32, 64) 30939 0.06% 99.95%
Mar 21 10:23:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:23:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:23:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:23:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:23:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:23:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:23:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:23:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:23:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:23:09  celestia[30304]:  --
Mar 21 10:23:50  celestia[30304]: 2024-03-21T10:23:50.005Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: stream reset
Mar 21 10:24:09  celestia[30304]: 2024-03-21T10:24:09.322Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27044357 miss: 705014319 keys-added: 53173045 keys-updated: 140371 keys-evicted: 53173019 cost-added: 556902427359335 cost-evicted: 556902159786023 sets-dropped: 1 sets-rejected: 454899906 gets-dropped: 27777280 gets-kept: 702884416 gets-total: 732058676 hit-ratio: 0.04
Mar 21 10:24:09  celestia[30304]: 2024-03-21T10:24:09.322Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:24:09  celestia[30304]:  -- Histogram:
Mar 21 10:24:09  celestia[30304]: Min value: 0
Mar 21 10:24:09  celestia[30304]: Max value: 30334
Mar 21 10:24:09  celestia[30304]: Count: 53165791
Mar 21 10:24:09  celestia[30304]: 50p: 2.00
Mar 21 10:24:09  celestia[30304]: 75p: 2.00
Mar 21 10:24:09  celestia[30304]: 90p: 2.00
Mar 21 10:24:09  celestia[30304]: [0, 2) 52065671 97.93% 97.93%
Mar 21 10:24:09  celestia[30304]: [2, 4) 566252 1.07% 99.00%
Mar 21 10:24:09  celestia[30304]: [4, 8) 274678 0.52% 99.51%
Mar 21 10:24:09  celestia[30304]: [8, 16) 149617 0.28% 99.79%
Mar 21 10:24:09  celestia[30304]: [16, 32) 52328 0.10% 99.89%
Mar 21 10:24:09  celestia[30304]: [32, 64) 30939 0.06% 99.95%
Mar 21 10:24:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:24:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:24:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:24:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:24:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:24:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:24:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:24:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:24:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:24:09  celestia[30304]:  --
Mar 21 10:24:12  celestia[30304]: 2024-03-21T10:24:12.297Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "EAA4140CD4CE12C94A2CD678BA9A2B49DD13B015E5140FF9E30CB46EF90ED0D3"}
Mar 21 10:24:12  celestia[30304]: 2024-03-21T10:24:12.980Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:24:35  celestia[30304]: 2024-03-21T10:24:35.588Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "D245DAB5978451B744E66C77799A4A1AE411F517977DAB66450D2FA899B6A029"}
Mar 21 10:24:44  celestia[30304]: 2024-03-21T10:24:44.364Z        INFO        canonical-log        swarm/swarm_listen.go:136        CANONICAL_PEER_STATUS: peer=12D3KooWDkc3G8t4oeH8JEXRdTTa1y76bijQEPeVzWAg6PWZSdCw addr=/ip4/101.43.72.56/udp/2121/quic-v1 sample_rate=100 connection_status="established" dir="inbound"
Mar 21 10:24:47  celestia[30304]: 2024-03-21T10:24:47.284Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWDqh9wLuVc74SFDqmXa4bgg2taMQ1vMsbyz4NAX9PktKD", "hash": "E446EDEAC51CB907B8111D8C5D67049787CFAEBE83FC4C46DBD39720E4F4071F"}
Mar 21 10:25:09  celestia[30304]: 2024-03-21T10:25:09.321Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27046963 miss: 705060286 keys-added: 53178075 keys-updated: 140390 keys-evicted: 53178058 cost-added: 556959553970909 cost-evicted: 556959296606273 sets-dropped: 1 sets-rejected: 454940824 gets-dropped: 27777280 gets-kept: 702933120 gets-total: 732107249 hit-ratio: 0.04
Mar 21 10:25:09  celestia[30304]: 2024-03-21T10:25:09.321Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:25:09  celestia[30304]:  -- Histogram:
Mar 21 10:25:09  celestia[30304]: Min value: 0
Mar 21 10:25:09  celestia[30304]: Max value: 30334
Mar 21 10:25:09  celestia[30304]: Count: 53170830
Mar 21 10:25:09  celestia[30304]: 50p: 2.00
Mar 21 10:25:09  celestia[30304]: 75p: 2.00
Mar 21 10:25:09  celestia[30304]: 90p: 2.00
Mar 21 10:25:09  celestia[30304]: [0, 2) 52070557 97.93% 97.93%
Mar 21 10:25:09  celestia[30304]: [2, 4) 566361 1.07% 99.00%
Mar 21 10:25:09  celestia[30304]: [4, 8) 274716 0.52% 99.51%
Mar 21 10:25:09  celestia[30304]: [8, 16) 149622 0.28% 99.79%
Mar 21 10:25:09  celestia[30304]: [16, 32) 52329 0.10% 99.89%
Mar 21 10:25:09  celestia[30304]: [32, 64) 30939 0.06% 99.95%
Mar 21 10:25:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:25:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:25:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:25:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:25:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:25:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:25:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:25:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:25:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:25:09  celestia[30304]:  --
Mar 21 10:25:22  celestia[30304]: 2024-03-21T10:25:22.980Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:26:09  celestia[30304]: 2024-03-21T10:26:09.321Z        WARN        badger4        [email protected]/db.go:460        Block cache might be too small. Metrics: hit: 27049497 miss: 705103753 keys-added: 53182488 keys-updated: 140405 keys-evicted: 53182472 cost-added: 557006066152991 cost-evicted: 557005809071079 sets-dropped: 1 sets-rejected: 454979863 gets-dropped: 27777280 gets-kept: 702979072 gets-total: 732153250 hit-ratio: 0.04
Mar 21 10:26:09  celestia[30304]: 2024-03-21T10:26:09.321Z        WARN        badger4        [email protected]/db.go:461        Cache life expectancy (in seconds):
Mar 21 10:26:09  celestia[30304]:  -- Histogram:
Mar 21 10:26:09  celestia[30304]: Min value: 0
Mar 21 10:26:09  celestia[30304]: Max value: 30334
Mar 21 10:26:09  celestia[30304]: Count: 53175244
Mar 21 10:26:09  celestia[30304]: 50p: 2.00
Mar 21 10:26:09  celestia[30304]: 75p: 2.00
Mar 21 10:26:09  celestia[30304]: 90p: 2.00
Mar 21 10:26:09  celestia[30304]: [0, 2) 52074798 97.93% 97.93%
Mar 21 10:26:09  celestia[30304]: [2, 4) 566489 1.07% 99.00%
Mar 21 10:26:09  celestia[30304]: [4, 8) 274754 0.52% 99.51%
Mar 21 10:26:09  celestia[30304]: [8, 16) 149627 0.28% 99.79%
Mar 21 10:26:09  celestia[30304]: [16, 32) 52331 0.10% 99.89%
Mar 21 10:26:09  celestia[30304]: [32, 64) 30939 0.06% 99.95%
Mar 21 10:26:09  celestia[30304]: [64, 128) 10703 0.02% 99.97%
Mar 21 10:26:09  celestia[30304]: [128, 256) 4921 0.01% 99.98%
Mar 21 10:26:09  celestia[30304]: [256, 512) 6321 0.01% 99.99%
Mar 21 10:26:09  celestia[30304]: [512, 1024) 2234 0.00% 100.00%
Mar 21 10:26:09  celestia[30304]: [1024, 2048) 1202 0.00% 100.00%
Mar 21 10:26:09  celestia[30304]: [2048, 4096) 640 0.00% 100.00%
Mar 21 10:26:09  celestia[30304]: [4096, 8192) 216 0.00% 100.00%
Mar 21 10:26:09  celestia[30304]: [8192, 16384) 57 0.00% 100.00%
Mar 21 10:26:09  celestia[30304]: [16384, 32768) 12 0.00% 100.00%
Mar 21 10:26:09  celestia[30304]:  --
Mar 21 10:26:20  celestia[30304]: 2024-03-21T10:26:20.890Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWR6SHsXPkkvhCRn6vp1RqSefgaT1X1nMNvrVjU2o3GoYy", "hash": "4252782158429AEAB4B7CA96A31C6DE2DB0C205D88BDE335D7B3503A09EA3BC8"}
Mar 21 10:26:32  celestia[30304]: 2024-03-21T10:26:32.979Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:26:58  systemd[1]: Stopping celestia-bridge...
Mar 21 10:26:59  celestia[30304]: 2024-03-21T10:26:59.058Z        INFO        rpc        rpc/server.go:108        server stopped
Mar 21 10:26:59  celestia[30304]: 2024-03-21T10:26:59.060Z        INFO        header/p2p        p2p/server.go:86        server: stopping server
Mar 21 10:26:59  celestia[30304]: 2024-03-21T10:26:59.095Z        WARN        core        core/listener.go:140        listener: unsubscribe        {"err": "context canceled"}
Mar 21 10:26:59  celestia[30304]: 2024-03-21T10:26:59.095Z        ERROR        core        core/listener.go:150        listener: resubscribe        {"err": "context canceled"}
Mar 21 10:26:59  celestia[30304]: 2024-03-21T10:26:59.096Z        WARN        shrex-sub        shrexsub/pubsub.go:83        unregistering topic        {"err": "no validator for topic mocha-4/eds-sub/v0.1.0"}
Mar 21 10:26:59  celestia[30304]: 2024-03-21T10:26:59.100Z        INFO        badger4        [email protected]/db.go:546        Lifetime L0 stalled for: 0s
Mar 21 10:26:59  celestia[30304]: 2024-03-21T10:26:59.238Z        INFO        badger4        [email protected]/db.go:625
Mar 21 10:26:59  celestia[30304]: Level 0 [ ]: NumTables: 01. Size: 11 MiB of 0 B. Score: 1.00->1.21 StaleData: 0 B Target FileSize: 64 MiB
Mar 21 10:26:59  celestia[30304]: Level 1 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Mar 21 10:26:59  celestia[30304]: Level 2 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Mar 21 10:26:59  celestia[30304]: Level 3 [B]: NumTables: 04. Size: 15 MiB of 18 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 4.0 MiB
Mar 21 10:26:59  celestia[30304]: Level 4 [ ]: NumTables: 31. Size: 177 MiB of 184 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 8.0 MiB
Mar 21 10:26:59  celestia[30304]: Level 5 [ ]: NumTables: 196. Size: 1.8 GiB of 1.8 GiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 16 MiB
Mar 21 10:26:59  celestia[30304]: Level 6 [ ]: NumTables: 933. Size: 18 GiB of 18 GiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 32 MiB
Mar 21 10:26:59  celestia[30304]: Level 7 [ ]: NumTables: 4151. Size: 180 GiB of 180 GiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 64 MiB
Mar 21 10:26:59  celestia[30304]: Level Done
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.852Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.852Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.852Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.852Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.852Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.852Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.852Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.853Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (local)
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.862Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: stream reset
Mar 21 10:27:00  celestia[30304]: 2024-03-21T10:27:00.883Z        INFO        badger4        [email protected]/db.go:546        Lifetime L0 stalled for: 0s
Mar 21 10:27:01  celestia[30304]: 2024-03-21T10:27:01.052Z        INFO        badger4        [email protected]/db.go:625
Mar 21 10:27:01  celestia[30304]: Level 0 [ ]: NumTables: 00. Size: 0 B of 0 B. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 16 MiB
Mar 21 10:27:01  celestia[30304]: Level 1 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Mar 21 10:27:01  celestia[30304]: Level 2 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Mar 21 10:27:01  celestia[30304]: Level 3 [ ]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Mar 21 10:27:01  celestia[30304]: Level 4 [B]: NumTables: 00. Size: 0 B of 10 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 2.0 MiB
Mar 21 10:27:01  celestia[30304]: Level 5 [ ]: NumTables: 18. Size: 58 MiB of 45 MiB. Score: 1.28->1.28 StaleData: 3.1 KiB Target FileSize: 4.0 MiB
Mar 21 10:27:01  celestia[30304]: Level 6 [ ]: NumTables: 100. Size: 449 MiB of 449 MiB. Score: 0.00->0.00 StaleData: 0 B Target FileSize: 8.0 MiB
Mar 21 10:27:01  celestia[30304]: Level Done
Mar 21 10:27:01  systemd[1]: celestia-bridge.service: Succeeded.
Mar 21 10:27:01  systemd[1]: Stopped celestia-bridge.
Mar 21 10:27:01  systemd[1]: Started celestia-bridge.
Mar 21 10:27:01  celestia[21805]: 2024-03-21T10:27:01.607Z        INFO        node        nodebuilder/module.go:26        Accessing keyring...
Mar 21 10:27:01  celestia[21805]: 2024-03-21T10:27:01.614Z        INFO        module/state        state/keyring.go:40        constructed keyring signer        {"backend": "test", "path": "/root/.celestia-bridge-mocha-4/keys", "key name": "BRIDGE_WALLET", "chain-id": "mocha-4"}
Mar 21 10:27:01  celestia[21805]: 2024-03-21T10:27:01.701Z        INFO        badger4        [email protected]/levels.go:171        All 118 tables opened in 67ms
Mar 21 10:27:01  celestia[21805]: 2024-03-21T10:27:01.701Z        INFO        badger4        [email protected]/discard.go:66        Discard stats nextEmptySlot: 41
Mar 21 10:27:01  celestia[21805]: 2024-03-21T10:27:01.701Z        INFO        badger4        [email protected]/db.go:368        Set nextTxnTs to 24088683
Mar 21 10:27:04  celestia[21805]: 2024/03/21 10:27:04 failed to sufficiently increase receive buffer size (was: 208 kiB, wanted: 2048 kiB, got: 416 kiB). See https://github.com/quic-go/quic-go/wiki/UDP-Buffer-Sizes for details.
Mar 21 10:27:05  celestia[21805]: 2024-03-21T10:27:05.380Z        INFO        canonical-log        swarm/swarm_dial.go:559        CANONICAL_PEER_STATUS: peer=12D3KooWA8GRTZLzxd3mZcceBiwmrBQgSmhKyrKXs5Zdwh2uwbqA addr=/ip4/167.235.13.19/udp/2121/quic-v1 sample_rate=100 connection_status="established" dir="outbound"
Mar 21 10:27:07  celestia[21805]: 2024-03-21T10:27:07.423Z        INFO        badger4        [email protected]/levels.go:118        3985 tables out of 5316 opened in 3.001s
Mar 21 10:27:08  celestia[21805]: 2024-03-21T10:27:08.334Z        INFO        badger4        [email protected]/levels.go:171        All 5316 tables opened in 3.913s
Mar 21 10:27:08  celestia[21805]: 2024-03-21T10:27:08.338Z        INFO        badger4        [email protected]/discard.go:66        Discard stats nextEmptySlot: 0
Mar 21 10:27:08  celestia[21805]: 2024-03-21T10:27:08.343Z        INFO        badger4        [email protected]/db.go:368        Set nextTxnTs to 834573
Mar 21 10:27:08  celestia[21805]: 2024-03-21T10:27:08.348Z        INFO        badger4        [email protected]/value.go:592        Deleting empty file: /root/.celestia-bridge-mocha-4/inverted_index//000001.vlog
Mar 21 10:27:11  celestia[21805]: 2024-03-21T10:27:11.968Z        INFO        canonical-log        swarm/swarm_dial.go:559        CANONICAL_PEER_STATUS: peer=12D3KooWRSbGkimKqHpwJVKm3C39tP9jddKi7TFQMVWSXyb8UUrQ addr=/ip4/79.186.144.13/tcp/55940 sample_rate=100 connection_status="established" dir="outbound"
Mar 21 10:27:14  celestia[21805]: 2024-03-21T10:27:14.334Z        INFO        basichost        basic/natmgr.go:112        DiscoverNAT error:no NAT found
Mar 21 10:27:16  celestia[21805]: 2024-03-21T10:27:16.634Z        INFO        share/discovery        discovery/discovery.go:307        discovered wanted peers        {"amount": 5}
Mar 21 10:27:23  celestia[21805]: 2024-03-21T10:27:23.129Z        INFO        header/p2p        p2p/subscriber.go:81        joining topic        {"topic ID": "/mocha-4/header-sub/v0.0.1"}
Mar 21 10:27:23  celestia[21805]: 2024-03-21T10:27:23.200Z        INFO        header/store        store/store.go:193        loaded head        {"height": 1430198, "hash": "F3DD6B105274082043D193D6EAF6BD715D40356BA73ACB57591C17AB3A6524C7"}
Mar 21 10:27:23  celestia[21805]: 2024-03-21T10:27:23.200Z        INFO        fraudserv        fraudserv/helpers.go:29        joining topic        {"id": "/mocha-4/fraud-sub/badencodingv0.1/v0.0.1"}
Mar 21 10:27:23  celestia[21805]: 2024-03-21T10:27:23.200Z        INFO        fraudserv        fraudserv/service.go:105        starting fraud proof service        {"protocol ID": "/mocha-4/fraud/v0.0.1"}
Mar 21 10:27:23  celestia[21805]: 2024-03-21T10:27:23.366Z        INFO        header/sync        sync/sync_head.go:138        new network head        {"height": 1430261, "hash": "966D5E3B8A793C7ED102EEA7E2128E35382E7D809AAD71B1A50C504CAE1D7C45"}
Mar 21 10:27:23  celestia[21805]: 2024-03-21T10:27:23.368Z        INFO        header/sync        sync/sync.go:216        syncing headers        {"from": 1430199, "to": 1430261}
Mar 21 10:27:23  celestia[21805]: 2024-03-21T10:27:23.490Z        INFO        header/p2p        p2p/server.go:77        server: listening for inbound header requests        {"protocol ID": "/mocha-4/header-ex/v0.0.3"}
Mar 21 10:27:23  celestia[21805]: 2024-03-21T10:27:23.490Z        INFO        rpc        rpc/server.go:90        server started        {"listening on": "127.0.0.1:26658"}
Mar 21 10:27:23  celestia[21805]: 2024-03-21T10:27:23.490Z        INFO        node        nodebuilder/node.go:113
Mar 21 10:27:23  celestia[21805]: /_____/  /_____/  /_____/  /_____/  /_____/
Mar 21 10:27:23  celestia[21805]: Started celestia DA node
Mar 21 10:27:23  celestia[21805]: node version:         v0.13.1
Mar 21 10:27:23  celestia[21805]: node type:         bridge
Mar 21 10:27:23  celestia[21805]: network:         mocha-4
Mar 21 10:27:23  celestia[21805]: /_____/  /_____/  /_____/  /_____/  /_____/
Mar 21 10:27:23  celestia[21805]: The p2p host is listening on:
Mar 21 10:27:23  celestia[21805]: *  /ip4/127.0.0.1/udp/2121/quic-v1/webtransport/certhash/uEiAQlWTjOisdOrne5-z7eEHIykeVJmBm9TeZa65Fsl7XPQ/certhash/uEiCNsurrUQH6g0TQb_0dAXOUbQ1tvx7VOGpNTBKh881XxA/p2p/12D3KooWKamHAWLsCvF1pek3GdPJNAw1VshEu7ZcJSDLEi1z83m6
Mar 21 10:27:23  celestia[21805]: *  /ip4/147.135.144.52/tcp/2121/p2p/12D3KooWKamHAWLsCvF1pek3GdPJNAw1VshEu7ZcJSDLEi1z83m6
Mar 21 10:27:23  celestia[21805]: *  /ip4/147.135.144.52/udp/2121/quic-v1/p2p/12D3KooWKamHAWLsCvF1pek3GdPJNAw1VshEu7ZcJSDLEi1z83m6
Mar 21 10:27:23  celestia[21805]: *  /ip4/147.135.144.52/udp/2121/quic-v1/webtransport/certhash/uEiAQlWTjOisdOrne5-z7eEHIykeVJmBm9TeZa65Fsl7XPQ/certhash/uEiCNsurrUQH6g0TQb_0dAXOUbQ1tvx7VOGpNTBKh881XxA/p2p/12D3KooWKamHAWLsCvF1pek3GdPJNAw1VshEu7ZcJSDLEi1z83m6
Mar 21 10:27:23  celestia[21805]: *  /ip6/::1/tcp/2121/p2p/12D3KooWKamHAWLsCvF1pek3GdPJNAw1VshEu7ZcJSDLEi1z83m6
Mar 21 10:27:23  celestia[21805]: *  /ip6/::1/udp/2121/quic-v1/p2p/12D3KooWKamHAWLsCvF1pek3GdPJNAw1VshEu7ZcJSDLEi1z83m6
Mar 21 10:27:23  celestia[21805]: *  /ip6/::1/udp/2121/quic-v1/webtransport/certhash/uEiAQlWTjOisdOrne5-z7eEHIykeVJmBm9TeZa65Fsl7XPQ/certhash/uEiCNsurrUQH6g0TQb_0dAXOUbQ1tvx7VOGpNTBKh881XxA/p2p/12D3KooWKamHAWLsCvF1pek3GdPJNAw1VshEu7ZcJSDLEi1z83m6
Mar 21 10:27:24  celestia[21805]: 2024-03-21T10:27:24.316Z        INFO        header/store        store/store.go:375        new head        {"height": 1430260, "hash": "02FFB01BA09BC333023525FA9AA7478F1F5E0572A8E69160DBCF033FAF1596E8"}
Mar 21 10:27:24  celestia[21805]: 2024-03-21T10:27:24.316Z        INFO        header/store        store/store.go:375        new head        {"height": 1430261, "hash": "966D5E3B8A793C7ED102EEA7E2128E35382E7D809AAD71B1A50C504CAE1D7C45"}
Mar 21 10:27:24  celestia[21805]: 2024-03-21T10:27:24.317Z        INFO        header/sync        sync/sync.go:234        finished syncing headers        {"from": 1430199, "to": 1430261, "elapsed time": 0.948749052}
Mar 21 10:27:31  celestia[21805]: 2024-03-21T10:27:31.295Z        WARN        shrex/eds        shrexeds/server.go:113        server: request hash not found        {"peer": "12D3KooWCUHPLqQXZzpTx1x3TAsdn3vYmTNDhzg66yG8hqoxGGN8", "hash": "635FA222B063E7CC1F3EDE9927ED4F80CD255324536DE8922DDF6CC64E71EF7D"}
Mar 21 10:27:31  celestia[21805]: 2024-03-21T10:27:31.380Z        INFO        header/store        store/store.go:375        new head        {"height": 1430262, "hash": "EE7962DE608962DAB26579DC7D7591B7C414FD9BE0498D77D5FD0857BFFF37CB"}
Mar 21 10:27:43  celestia[21805]: 2024-03-21T10:27:43.159Z        INFO        header/store        store/store.go:375        new head        {"height": 1430263, "hash": "524D64A188C71F8A34F7A5A76E63249068B84231CE2CFA34E1EF9E24DB721665"}
Mar 21 10:27:55  celestia[21805]: 2024-03-21T10:27:55.490Z        INFO        header/store        store/store.go:375        new head        {"height": 1430264, "hash": "FC71F4FBF0783B24B1EE3A82769B56E08210E31258761F19853C299C746F1624"}
Mar 21 10:28:04  celestia[21805]: 2024-03-21T10:28:04.267Z        INFO        badger4        [email protected]/value.go:1035        Found value log max discard fid: 41 discard: 183546042
Mar 21 10:28:04  celestia[21805]: 2024-03-21T10:28:04.267Z        INFO        badger4        [email protected]/value.go:184        Rewriting fid: 41
Mar 21 10:28:07  celestia[21805]: 2024-03-21T10:28:07.076Z        INFO        header/store        store/store.go:375        new head        {"height": 1430265, "hash": "E9E9F5199B47DDDF98483075ADDBC24C0972D2FB7C466DAB36C1284C1BA18292"}
Mar 21 10:28:17  celestia[21805]: 2024-03-21T10:28:17.480Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:28:17  celestia[21805]: 2024-03-21T10:28:17.573Z        INFO        badger4        [email protected]/value.go:336        Processed 85 entries in 1 loops
Mar 21 10:28:17  celestia[21805]: 2024-03-21T10:28:17.573Z        INFO        badger4        [email protected]/value.go:337        Total entries: 97209. Moved: 49294
Mar 21 10:28:17  celestia[21805]: 2024-03-21T10:28:17.573Z        INFO        badger4        [email protected]/value.go:338        Removing fid: 41
Mar 21 10:28:17  celestia[21805]: 2024-03-21T10:28:17.969Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: Application error 0x0 (remote)
Mar 21 10:28:20  celestia[21805]: 2024-03-21T10:28:20.072Z        INFO        bitswap-server        server/server.go:547        Bitswap Client ReceiveError: stream reset
Mar 21 10:28:20  celestia[21805]: 2024-03-21T10:28:20.155Z        INFO        header/store        store/store.go:375        new head        {"height": 1430266, "hash": "1130B413EC3E59F9FF3356D817FFCCBDE298835AAFD5D8388901342B01706F3F"}

Mar 21 10:28:41  celestia[21805]: 2024-03-21T10:28:41.590Z        INFO        header/store        store/store.go:375        new head        {"height": 1430267, "hash": "C1D2BBB48D6469551095A2625193368EBEA1CFA35889F90692F78AB70F51395A"}
Mar 21 10:28:53  celestia[21805]: 2024-03-21T10:28:53.133Z        INFO        header/store        store/store.go:375        new head        {"height": 1430268, "hash": "F91D0844F8ED030EA48FE6C05E2F53919F3A0EC204B5D5C761D7A41B8E559B6B"}
Mar 21 10:29:04  celestia[21805]: 2024-03-21T10:29:04.783Z        INFO        header/store        store/store.go:375        new head        {"height": 1430269, "hash": "D1E7CDF0E2521D65818812714BB4E8B184B27E53D2FFB7EC94E2F482DC32D982"}
Mar 21 10:29:17  celestia[21805]: 2024-03-21T10:29:17.123Z        INFO        header/store        store/store.go:375        new head        {"height": 1430270, "hash": "9EB91F1DDF16BD322092E3603782ED9F2440101DB5917E40FF3467015373C004"}

Notes

No response

shivlim avatar Mar 21 '24 10:03 shivlim

//bump

we also got our mocha bridge node stuck while seeing the same messages in the logs. restart resolved it tho.

SimonKraus avatar Jun 17 '24 07:06 SimonKraus

Hi @shivlim @SimonKraus, which core endpoint are you using for the BN to sync off of? Is it a public RPC?

renaynay avatar Jun 18 '24 07:06 renaynay

Hey @renaynay - we're seeing it in different combinations (own rpc node on mainnet, public one on testnet)

SimonKraus avatar Jun 18 '24 07:06 SimonKraus

@SimonKraus you passed a core endpoint of a mainnet node to the mocha BN ?

renaynay avatar Jun 18 '24 07:06 renaynay

No , sorry - of course not.

I meant we are seeing this on our Mainnet bridge, which uses a dedicated self-hosted RPC, as well as on our Mocha bridge, which uses a public mocha RPC.

SimonKraus avatar Jun 18 '24 07:06 SimonKraus

@SimonKraus we're investigating now.

renaynay avatar Jun 18 '24 08:06 renaynay

@SimonKraus what DA node version are you on? I see this issue was open March 21 so I need a bit more details since it's a few months / releases past

same for @shivlim ^

renaynay avatar Jun 19 '24 09:06 renaynay

We are on 0.13.7

Sorry for the late reply

SimonKraus avatar Jun 20 '24 20:06 SimonKraus

Closing because Shwap addressed such issues. Thanks!

cristaloleg avatar Jan 21 '25 13:01 cristaloleg