lnd icon indicating copy to clipboard operation
lnd copied to clipboard

Peers are still unable to connect after IP address change

Open viaj3ro opened this issue 3 years ago • 4 comments

I've reported this issue a few times already. According to https://github.com/lightningnetwork/lnd/issues/5377 it got fixed quite a while ago, but it still persists, as reported again in January: https://github.com/lightningnetwork/lnd/issues/6128 and also from another user who is running CLN: https://github.com/ElementsProject/lightning/issues/5037

Unfortunately the issue still seems to be very widespread. I had another IP address change and almost 40 tor peers are unable to connect. That's almost 25% of my ~170 peers: image

Even 1ml.com didn't pickup on my new IP and I talked to Alex Bosworth and he said he is running custom software to make sure yalls.org isn't affected by this specific issue.

After changing my alias twice, 1ml finally got the new IP, but the ~40 tor peers are still stuck offline.

My node is SilentBob (02e9046555a9665145b0dbd7f135744598418df7d61d3660659641886ef1274844)

viaj3ro avatar Sep 17 '22 12:09 viaj3ro

hi @viaj3ro - quick question just to try to narrow down where the problem may be: if you ask one of your peers who cant reconnect to you to do an lncli getnodeinfo for your node id, can they see your new IP listed there?

ellemouton avatar Sep 19 '22 06:09 ellemouton

I also experienced this issue, and only with SilentBobs help was I able to fix and reconnect using new IP.

pfergi42 avatar Sep 19 '22 20:09 pfergi42

@ellemouton I'm pretty sure they use Eclair.

I get getnodeinfo on a mainnet node and it shows a dated (?), different IP from that of 1 ML.

Ultimately, what I think is the issue here is the propagation of node announcements across the network, which may run into the set of heterogenous throttling and rebroadcast mechanisms of each of the implementations have.

They also mention they see this issue w/ Tor peers. IIUC, their node is a clearnet-only node, so it can't connect to the Tor nodes. Instead the Tor nodes must connect to it, since they only have an onion service address.

Roasbeef avatar Sep 19 '22 22:09 Roasbeef

@Roasbeef yes, I'm running eclair and the issue is with tor and clearnet peers, but the clearnet peers I can connect back manually myself (as long as their IP announcement is correct).

@ellemouton I talked with @pfergi42 and they never got my new IP. Even after many days. Last time I got a new IP I ended up force closing quite a few channels with tor peers, since even after many months they never got the new IP.

Sometimes changing the alias helps, but usually only for a small number of unconnected peers.

@Roasbeef I understand that there are propagation issues but shouldn't there be a feasible way for nodes to be able to somehow get an up to date IP announcement from their own peers? especially if channels are stuck offline for a while?

viaj3ro avatar Sep 20 '22 11:09 viaj3ro

I understand that there are propagation issues but shouldn't there be a feasible way for nodes to be able to somehow get an up to date IP announcement from their own peers? especially if channels are stuck offline for a while?

The nature of the gossip network is that they'll eventually get this information. On the spec side, we recently added support to advertise a domain name instead of just an IP. Nodes can then use DNS instead of the gossip network to be able to obtain the latest IP for a node. I'm not sure if Eclair has implemented this yet, but lnd will propagate these announcements. If you have a dynamic IP, then the best way is just to use the domain name instead.

Roasbeef avatar Sep 28 '22 02:09 Roasbeef

the problem is, that 'eventually' in practice just seems to be never for a big chunk of (tor) nodes. Since my IP is semi dynamic (changes once or twice a year on average) I'd actually prefer to not introduce more potential failure modes via DNS. If that changes for some reason, the issue will be exactly the same. Why not solve it once and for all?

viaj3ro avatar Oct 04 '22 14:10 viaj3ro