lnd icon indicating copy to clipboard operation
lnd copied to clipboard

Opened Channel and Appearing as Zombie

Open CryptoHig opened this issue 3 years ago • 24 comments

Background

I opened a channel last night. It doesn't show any fees. In Thunderhub, when I try to edit the fees, it returns the following error:

"Error getting channel information. Try refreshing the page."

I've tried refreshing, I've tried restarting node, same results.

The log returns:

"2022-08-25 21:48:14.764 [ERR] RPCS: [/lnrpc.Lightning/GetChanInfo]: edge marked as zombie"

Your environment

  • version of lnd: 0.15.0-beta
  • which operating system (uname -a on *Nix): Voltage.Cloud
  • version of btcd, bitcoind, or other backend: Unsure
  • any other relevant environment details

Steps to reproduce

Unsure as node is hosted via Voltage

Expected behaviour

It should have opened a screen for me to set fees.

Actual behaviour

Produced a Zombie error.

CryptoHig avatar Aug 25 '22 23:08 CryptoHig

So typically a channel only shows as a "zombie" after it's been open for 2 weeks, but hasn't sent any channel updates. Given that, I think these errors might be uncorrelated.

Was the node offline for a long time?

If you have CLI access, are you able to do lncli getchaninfo <scid> locally to see if your node knows of the channel?

Do you have any logs for around the time of the channel open?

Roasbeef avatar Aug 26 '22 06:08 Roasbeef

It's definitely that channel that is marked as a zombie. I just tried to edit fees twice:

2022-08-26 12:31:15.933 [ERR] RPCS: [/lnrpc.Lightning/GetChanInfo]: edge marked as zombie 2022-08-26 12:31:12.628 [ERR] RPCS: [/lnrpc.Lightning/GetChanInfo]: edge marked as zombie

I had a channel open to them previously and it was closed out a few weeks ago. Maybe that has something to do with it?

CryptoHig avatar Aug 26 '22 12:08 CryptoHig

what chain are you on

Crypt-iQ avatar Aug 26 '22 12:08 Crypt-iQ

Clearnet, I think?

CryptoHig avatar Aug 26 '22 13:08 CryptoHig

I mean mainnet or testnet. do you have debug logs youd be willing to share

Crypt-iQ avatar Aug 26 '22 14:08 Crypt-iQ

Sorry about that. Mainnet. I'll send you logs, now.

CryptoHig avatar Aug 26 '22 14:08 CryptoHig

Emailed logs.

CryptoHig avatar Aug 26 '22 14:08 CryptoHig

I mean mainnet or testnet. do you have debug logs youd be willing to share

Just making sure you received the logs @Crypt-iQ

CryptoHig avatar Aug 28 '22 13:08 CryptoHig

Yes I've received the logs, thanks

Crypt-iQ avatar Aug 29 '22 14:08 Crypt-iQ

I am the other party

On my end I see the channel on my node but it does not appear in graph (hence not visible anywhere). On my end I am able to set fees but cannot route through this channel because it is not on graph.

lncli getchaninfo returns node2 policy as null (which implies my peers is unable to setup fees on this channel)

umbrel@umbrel:~/utils $ lncli getchaninfo 825706844218261505
{
    "channel_id": "825706844218261505",
    "chan_point": "1bae01e863444d89f5d3612ed27adc5f1553396f16feb774a0dbba93d7632805:1",
    "last_update": 1661478013,
    "node1_pub": "03c5528c628681aa17ab9e117aa3ee6f06c750dfb17df758ecabcd68f1567ad8c1",
    "node2_pub": "03f64f9cba205695b89f0208f0326d09c45e6ebf104f79f73eb204dfa2c0badee5",
    "capacity": "8000000",
    "node1_policy": {
        "time_lock_delta": 144,
        "min_htlc": "1000",
        "fee_base_msat": "9999000",
        "fee_rate_milli_msat": "966",
        "disabled": false,
        "max_htlc_msat": "7920000000",
        "last_update": 1661478013
    },
    "node2_policy": null
}

I can also see the channel in lncli listchannel

{
  "active": true,
  "remote_pubkey": "03f64f9cba205695b89f0208f0326d09c45e6ebf104f79f73eb204dfa2c0badee5",
  "channel_point": "1bae01e863444d89f5d3612ed27adc5f1553396f16feb774a0dbba93d7632805:1",
  "chan_id": "825706844218261505",
  "capacity": "8000000",
  "local_balance": "0",
  "remote_balance": "7998160",
  "commit_fee": "1510",
  "commit_weight": "772",
  "fee_per_kw": "1050",
  "unsettled_balance": "0",
  "total_satoshis_sent": "0",
  "total_satoshis_received": "0",
  "num_updates": "21",
  "pending_htlcs": [],
  "csv_delay": 961,
  "private": false,
  "initiator": false,
  "chan_status_flags": "ChanStatusDefault",
  "local_chan_reserve_sat": "80000",
  "remote_chan_reserve_sat": "80000",
  "static_remote_key": false,
  "commitment_type": "ANCHORS",
  "lifetime": "98220",
  "uptime": "98149",
  "close_address": "",
  "push_amount_sat": "0",
  "thaw_height": 0,
  "local_constraints": {
    "csv_delay": 961,
    "chan_reserve_sat": "80000",
    "dust_limit_sat": "354",
    "max_pending_amt_msat": "7920000000",
    "min_htlc_msat": "1",
    "max_accepted_htlcs": 483
  },
  "remote_constraints": {
    "csv_delay": 961,
    "chan_reserve_sat": "80000",
    "dust_limit_sat": "354",
    "max_pending_amt_msat": "7920000000",
    "min_htlc_msat": "1",
    "max_accepted_htlcs": 21
  },
  "alias_scids": [],
  "zero_conf": false,
  "zero_conf_confirmed_scid": "0"
}

BhaagBoseDK avatar Aug 29 '22 14:08 BhaagBoseDK

since you are using voltage.cloud, is voltage running the node for you on their servers with their own full-node backends?

Crypt-iQ avatar Aug 29 '22 17:08 Crypt-iQ

since you are using voltage.cloud, is voltage running the node for you on their servers with their own full-node backends?

That is correct. I think. I have their standard full node plan:

Standard Node Backed by a Bitcoin full node on a shared platform with Tor only

CryptoHig avatar Aug 29 '22 17:08 CryptoHig

Just following up on this. I can't co-op close, force close, set fees, rebalance, nothing ... Thank you!

CryptoHig avatar Sep 02 '22 19:09 CryptoHig

Screen Shot 2022-08-31 at 2 56 14 PM

This is the error it gives me.

CryptoHig avatar Sep 02 '22 19:09 CryptoHig

@itshig this error shown above sounds like an issue with the UI (RTL or Voltage UI?) itself, trying to fetch channel information to display, then not showing the buttons because of the error. You should be able to close the channel with lncli (if you choose to do so). We're still trying to find out what exactly happened though.

guggero avatar Sep 09 '22 11:09 guggero

@itshig this error shown above sounds like an issue with the UI (RTL or Voltage UI?) itself, trying to fetch channel information to display, then not showing the buttons because of the error. You should be able to close the channel with lncli (if you choose to do so). We're still trying to find out what exactly happened though.

Thanks, I'll try to close it via lncli sometime this weekend. Have a really busy day today.

CryptoHig avatar Sep 09 '22 12:09 CryptoHig

The issue is that the funding manager receives confirmations, but then the router can't find the funding tx with btcd, meaning it's marked as a zombie

Crypt-iQ avatar Sep 12 '22 14:09 Crypt-iQ

@itshig this error shown above sounds like an issue with the UI (RTL or Voltage UI?) itself, trying to fetch channel information to display, then not showing the buttons because of the error. You should be able to close the channel with lncli (if you choose to do so). We're still trying to find out what exactly happened though.

What is the proper way to close a channel via lncli? I've tried it about every way I can think of:

 lncli closechannel G-Spot-21_69_420 --sat_per_vbyte 1 [lncli] rpc error: code = Unknown desc = encoding/hex: invalid byte: U+0047 'G'  lncli closechannel ⚡G-Spot-21_69_420⚡ --sat_per_vbyte 1 [lncli] rpc error: code = Unknown desc = encoding/hex: invalid byte: U+00E2 'â'  lncli closechannel 809918956902809601 --sat_per_vbyte 1 [lncli] rpc error: code = Unknown desc = channel not found  lncli closechannel 736617x2828x1 --sat_per_vbyte 1 [lncli] rpc error: code = Unknown desc = encoding/hex: invalid byte: U+0078 'x' 

CryptoHig avatar Sep 12 '22 19:09 CryptoHig

You need to split the channel_point into its two parts:

lncli closechannel --funding_txid 1bae01e863444d89f5d3612ed27adc5f1553396f16feb774a0dbba93d7632805 --output_index 1

guggero avatar Sep 12 '22 20:09 guggero

Thank you! This worked! Where did you get the funding_txid from? This is all confusing to me lol.

CryptoHig avatar Sep 12 '22 20:09 CryptoHig

Also, would this work on a channel that is stuck in closing? I have another channel that has been stuck for a few months that I can't seem to figure out how to get it to move out of pending close. Screen Shot 2022-09-12 at 4 24 04 PM

CryptoHig avatar Sep 12 '22 20:09 CryptoHig

Thank you! This worked! Where did you get the funding_txid from? This is all confusing to me lol.

You can get the channel_point (funding transaction id + output index) from the listchannels command.

Also, would this work on a channel that is stuck in closing? I have another channel that has been stuck for a few months that I can't seem to figure out how to get it to move out of pending close.

It looks like this channel is still open. If your peer is offline, you can try to force close the channel with the --force flag (see lncli closechannel -h). Note that this incurs higher transaction fees for the one who opened the channel than for a normal close.

bitromortac avatar Sep 13 '22 06:09 bitromortac

I really appreciate the help. I think all of my issues are in the process of being rectified. This being the last of them:

 lncli closechannel --funding_txid 16932e23337fa01183be57c92a48a2d1b8e4bf5818ebee271c996f88c121c042 --conf_target 1 [lncli] rpc error: code = Unknown desc = unable to gracefully close channel while peer is offline (try force closing it instead): channel link not found  lncli closechannel --funding_txid 16932e23337fa01183be57c92a48a2d1b8e4bf5818ebee271c996f88c121c042 --force { "closing_txid": "db462cf8fd432a207c898eb5b3ad4c75e198026ba9965b4696ef6e8f28277eab" } 

CryptoHig avatar Sep 14 '22 00:09 CryptoHig

I really appreciate the help. I think all of my issues are in the process of being rectified. This being the last of them:

 lncli closechannel --funding_txid 16932e23337fa01183be57c92a48a2d1b8e4bf5818ebee271c996f88c121c042 --conf_target 1 [lncli] rpc error: code = Unknown desc = unable to gracefully close channel while peer is offline (try force closing it instead): channel link not found  lncli closechannel --funding_txid 16932e23337fa01183be57c92a48a2d1b8e4bf5818ebee271c996f88c121c042 --force { "closing_txid": "db462cf8fd432a207c898eb5b3ad4c75e198026ba9965b4696ef6e8f28277eab" } 

It looks like that's the expected outcome.

In general, make sure to also explicitly specify the output index --output_index so you won't accidentally close a channel should multiple channels have been opened in a batch (if the output index is zero, it's not strictly needed).

bitromortac avatar Sep 14 '22 07:09 bitromortac