Libp2pPort resumes to the initial Store on failures

[rodrigo-o]

When something makes Libp2pPort to crash, right now it goes back to the initial paramters set at node startup which cause current canonical blocks to be marked as invalid because of being in the future or even failing to retrieve a particular block in the past that has already been pruned. An example:

2024-10-06 02:33:12.006	[Libp2p] Slot transition
2024-10-06 02:33:00.585	[Fork choice] Failed to add block: block is from the future
2024-10-06 02:33:00.585	[PendingBlocks] Saving block as invalid after ForkChoice.on_block/2 error: block is from the future
2024-10-06 02:33:00.529	[Optimistic Sync] Waiting 15.0 seconds to discover some peers before requesting blocks.
2024-10-06 02:33:00.529	[Fork choice] Adding new block
2024-10-06 02:33:00.468	Last message: {#Port<0.15>, {:data, <<10, 242, 74, 10, 40, 47, 101, 116, 104, 50, 47, 100, 51, 49, 102, 54, 49, 57, 49, 47, 98, 108, 111, 98, 95, 115, 105, 100, 101, 99, 97, 114, 95, 49, 47, 115, 115, 122, 95, 115, 110, 97, 112, 112, 121, 26, ...>>}}" mfa=:gen_server.error_info/8 pid=#PID<0.444.0>
2024-10-06 02:33:00.468	    (elixir 1.16.2) lib/stream.ex:1828: Enumerable.Stream.do_each/4
2024-10-06 02:33:00.468	    (elixir 1.16.2) lib/enum.ex:4839: Enumerable.List.reduce/3
2024-10-06 02:33:00.468	    (elixir 1.16.2) lib/stream.ex:1816: anonymous fn/3 in Enumerable.Stream.reduce/3
2024-10-06 02:33:00.468	    (elixir 1.16.2) lib/enum.ex:4396: anonymous fn/3 in Enum.reduce/3
2024-10-06 02:33:00.468	    (lambda_ethereum_consensus 0.1.0) lib/lambda_ethereum_consensus/fork_choice/head.ex:50: anonymous fn/6 in LambdaEthereumConsensus.ForkChoice.Head.get_weight/3
2024-10-06 02:33:00.468	    (elixir 1.16.2) lib/map.ex:450: Map.put_new_lazy/3
2024-10-06 02:33:00.468	    (lambda_ethereum_consensus 0.1.0) lib/types/store.ex:120: Types.Store.get_ancestor/3
2024-10-06 02:33:00.468	    (lambda_ethereum_consensus 0.1.0) lib/lambda_ethereum_consensus/store/blocks.ex:72: LambdaEthereumConsensus.Store.Blocks.get_block!/1
2024-10-06 02:33:00.468	** (RuntimeError) Block not found: 0x75e7d39a4266b27fb6260f64e2903e29e7de4fd0cfbc3af8e58fa28f7aaa0afa
2024-10-06 02:33:00.468	ts=2024-10-06T05:33:00.468 level=error msg="GenServer LambdaEthereumConsensus.Libp2pPort terminating
2024-10-06 02:33:00.446	[Fork choice] Block processed. Recomputing head.
2024-10-06 02:33:00.226	[Fork choice] Adding new block
2024-10-06 02:33:00.146	[Gossip] Block received, block.slot: 6038265.
2024-10-06 02:33:00.017	[Libp2p] Slot transition
2024-10-06 02:32:49.096	[Fork choice] Recomputed head
2024-10-06 02:32:49.096	[Fork choice] Added new block
2024-10-06 02:32:48.814	[Fork choice] Block processed. Recomputing head.
2024-10-06 02:32:48.625	[Fork choice] Adding new block

As seen after a failure it tries to readd the same block that failed previously and instead of failing again it just mark it as invalid, effectively detaining state transitions while maintaining the Node still running.