mina icon indicating copy to clipboard operation
mina copied to clipboard

[Error] In uptime service, POST of uptime data was interrupted

Open xOctave opened this issue 4 years ago • 2 comments

Preliminary Checks

  • [X] This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/MinaProtocol/mina/issues
  • [X] This issue is not a question, feature request, RFC, or anything other than a bug report. Please post those things in GitHub Discussions: https://github.com/MinaProtocol/mina/discussions

Description

Oct 28 07:03:00 ip-172-31-80-92 mina[5038]: 2021-10-28 07:03:00 UTC [Error] In uptime service, POST of uptime data was interrupted

with 10 lines of context, before/after

Oct 28 07:00:18 ip-172-31-80-92 mina[5038]: 2021-10-28 07:00:18 UTC [Warn] Not rebroadcasting block $state_hash because it was received "1 slots too late"
Oct 28 07:00:18 ip-172-31-80-92 mina[5038]:         state_hash: "3NKa6bxozVtPgmCV7De2Q7G2LhBafego4roK2hNn7T4BKwmnSFMj"
Oct 28 07:00:18 ip-172-31-80-92 mina[5038]: 2021-10-28 07:00:18 UTC [Info] Saw block with state hash $state_hash
Oct 28 07:00:18 ip-172-31-80-92 mina[5038]:         state_hash: "3NKa6bxozVtPgmCV7De2Q7G2LhBafego4roK2hNn7T4BKwmnSFMj"
Oct 28 07:02:12 ip-172-31-80-92 mina[5038]: 2021-10-28 07:02:12 UTC [Info] verifier successfully got sigkill
Oct 28 07:02:12 ip-172-31-80-92 mina[5038]: 2021-10-28 07:02:12 UTC [Info] verifier successfully stopped
Oct 28 07:02:12 ip-172-31-80-92 mina[5038]: 2021-10-28 07:02:12 UTC [Info] Starting a new verifier process
Oct 28 07:02:41 ip-172-31-80-92 mina[5038]: 2021-10-28 07:02:41 UTC [Info] Daemon started process of kind "Verifier" with pid 6520
Oct 28 07:03:00 ip-172-31-80-92 mina[5038]: 2021-10-28 07:03:00 UTC [Info] Uptime service will attempt to send a block and SNARK work
Oct 28 07:03:00 ip-172-31-80-92 mina[5038]: 2021-10-28 07:03:00 UTC [Error] In uptime service, POST of uptime data was interrupted
Oct 28 07:04:17 ip-172-31-80-92 mina[5038]: 2021-10-28 07:04:16 UTC [Error] Error when sending block with state hash $state_hash to uptime service at URL "http://34.134.227.208/v1/submit"
Oct 28 07:04:17 ip-172-31-80-92 mina[5038]:         state_hash: "3NLeFvH8RqNM4ZqhJfYdm6r2Eh2M123zpdRnECHZYxjRitm9mqzm"
Oct 28 07:04:33 ip-172-31-80-92 mina[5038]: 2021-10-28 07:04:33 UTC [Warn] validation callback timed out before we could respond
Oct 28 07:04:33 ip-172-31-80-92 mina[5038]: 2021-10-28 07:04:33 UTC [Error] validation timed out :(
Oct 28 07:04:37 ip-172-31-80-92 mina[5038]: 2021-10-28 07:04:37 UTC [Warn] validation callback timed out before we could respond
Oct 28 07:04:37 ip-172-31-80-92 mina[5038]: 2021-10-28 07:04:37 UTC [Error] validation timed out :(
Oct 28 07:06:57 ip-172-31-80-92 mina[5038]: 2021-10-28 07:06:57 UTC [Info] Received a block from $sender
Oct 28 07:06:57 ip-172-31-80-92 mina[5038]:         sender: {
Oct 28 07:06:57 ip-172-31-80-92 mina[5038]:   "Remote": {
Oct 28 07:06:57 ip-172-31-80-92 mina[5038]:     "host": "142.93.100.73",
Oct 28 07:06:57 ip-172-31-80-92 mina[5038]:     "peer_id": "12D3KooWPmjJGp9HEk9uyEJ9RyrLpgGQE2QrQj23gRGdU1EQYdDk",

log export in https://s3.amazonaws.com/mina.crowbar.systems/mainnet/export/2021-10-28_07-18-20.tar.gz

Steps to Reproduce

  1. Running 1.2.0 on Ubuntu
  2. Uptime service set to http://34.134.227.208/v1/submit

Expected Result

Not having this kind of entry in the log :)

Actual Result

Having this kind of entry in the log

How frequently do you see this issue?

Sometimes

What is the impact of this issue on your ability to run a node?

Blocker

Status

Mina daemon status
-----------------------------------

Global number of accounts:                     68555
Block height:                                  75165
Max observed block height:                     75165
Max observed unvalidated block height:         75165
Local uptime:                                  6h6m9s
Ledger Merkle root:                            jwJnd2gh8nj2pr9xtZAz3Ty7zWFTrappGuzcAaDKe2Mwh3LagZD
Protocol state hash:                           3NLfVau9AYWhJ3qmLJQYcPu5TFZUvMXS73AozLzfV3Ey9YxSR1vR
Chain id:                                      5f704cc0c82e0ed70e873f0893d7e06f148524e3f0bdae2afb02e7819a0c24d1
Git SHA-1:                                     fe51f1ef5cfbcb4d5f559af39080963cf28ff1af
Configuration directory:                       /home/ubuntu/.mina-config
Peers:                                         54
User_commands sent:                            0
SNARK worker:                                  None
SNARK work fee:                                100000000
Sync status:                                   Synced
Catchup status:                                
	To build breadcrumb:           0
	To initial validate:           0
	Finished:                      304
	To verify:                     0
	To download:                   0
	Waiting for parent to finish:  0

Block producers running:                       1 (B62qrcQDeAfg2qSnZhdF8wQ78G9QmKNDqpESVt2SL43F97kF3Cyv32a)
Coinbase receiver:                             Block producer
Best tip consensus time:                       epoch=15, slot=1046
Best tip global slot (across all hard-forks):  108146
Next block will be produced in:                in 13.674h for slot: 108420 slot-since-genesis: 108420 (Generated from consensus at slot: 108064 slot-since-genesis: 108064)
Consensus time now:                            epoch=15, slot=1046
Consensus mechanism:                           proof_of_stake
Consensus configuration:                       
	Delta:                     0
	k:                         290
	Slots per epoch:           7140
	Slot duration:             3m
	Epoch duration:            14d21h
	Chain start timestamp:     2021-03-17 00:00:00.000000Z
	Acceptable network delay:  3m

Addresses and ports:                           
	External IP:    3.84.79.225
	Bind IP:        0.0.0.0
	Libp2p PeerID:  12D3KooWQBiq7tzFZ2uScLDrXvsk2ecTzWp3Kj7pFvQ5YKLXxPhT
	Libp2p port:    8302
	Client port:    8301

Additional information

You can get in touch with me on Discord.

xOctave avatar Oct 28 '21 07:10 xOctave

Just adding a comment since no activity here - want to register that I also see these daily -- 2 - 4 errors every day.

jrwashburn avatar Dec 14 '21 17:12 jrwashburn

@hsq125 @jrwashburn DO you still experience same issue?

shimkiv avatar Sep 20 '22 14:09 shimkiv

Closing this issue because of being inactive for long time. Please feel free to open another ticket should you experience any further issues.

shimkiv avatar Sep 30 '22 13:09 shimkiv