stabilizer icon indicating copy to clipboard operation
stabilizer copied to clipboard

Device panic with static IP and incorrect broker address

Open svrotter opened this issue 3 years ago • 13 comments

When specifying a static IP address in the firmware build command but having a incorrect broker address at the same time, the program panics at assertion failed: protocol_addr.is_unicast(). If both addresses are correct, everything works fine though.

svrotter avatar Feb 16 '22 16:02 svrotter

@svrotter Can you please provide what STATIC_IP you were using when this occurred?

Also, please provide the full backtrace for the panic if possible to aid in debugging.

ryan-summers avatar Feb 16 '22 16:02 ryan-summers

INFO - EUI48: 04-91-62-d9-4c-7f
INFO - setup() complete
WARN - Network link DOWN
panicked at 'assertion failed: protocol_addr.is_unicast()', C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\smoltcp-0.8.0\src\iface\neighbor.rs:199:9
────────────────────────────────────────────────────────────────────────────────
(HOST) WARN  no RAM region appears to contain the stack; cannot determine if this was a stack overflow
stack backtrace:
   0: HardFaultTrampoline
      <exception entry>
   1: lib::inline::__udf
        at ./asm/inline.rs:181:5
   2: __udf
        at ./asm/lib.rs:51:17
   3: cortex_m::asm::udf
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\cortex-m-0.7.4\src\asm.rs:43:5
   4: rust_begin_unwind
        at src\hardware\mod.rs:93:5
   5: core::panicking::panic_fmt
        at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panicking.rs:100:14
   6: core::panicking::panic
        at /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/core/src/panicking.rs:50:5
   7: smoltcp::iface::neighbor::Cache::lookup
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\smoltcp-0.8.0\src\iface\neighbor.rs:199:9
   8: smoltcp::iface::interface::InterfaceInner::lookup_hardware_addr
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\smoltcp-0.8.0\src\iface\interface.rs:2311:15
   9: smoltcp::iface::interface::InterfaceInner::dispatch_ip
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\smoltcp-0.8.0\src\iface\interface.rs:2398:59
  10: smoltcp::iface::interface::Interface<DeviceT>::socket_egress::{{closure}}
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\smoltcp-0.8.0\src\iface\interface.rs:934:21
  11: smoltcp::socket::tcp::TcpSocket::dispatch
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\smoltcp-0.8.0\src\socket\tcp.rs:2312:9
  12: smoltcp::iface::interface::Interface<DeviceT>::socket_egress
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\smoltcp-0.8.0\src\iface\interface.rs:933:40
  13: smoltcp::iface::interface::Interface<DeviceT>::poll
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\smoltcp-0.8.0\src\iface\interface.rs:750:31
  14: smoltcp_nal::NetworkStack<DeviceT,Clock>::poll
        at C:\Users\Svoe\.cargo\git\checkouts\smoltcp-nal-f2ec3ae73a86846c\8d2c48e\src\lib.rs:205:23
  15: stabilizer::net::network_processor::NetworkProcessor::update::{{closure}}
        at src\net\network_processor.rs:67:39
  16: smoltcp_nal::shared::NetworkStackProxy<S>::lock::{{closure}}
        at C:\Users\Svoe\.cargo\git\checkouts\smoltcp-nal-f2ec3ae73a86846c\8d2c48e\src\shared.rs:44:33
  17: <shared_bus::mutex::AtomicCheckMutex<BUS> as shared_bus::mutex::BusMutex>::lock
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\shared-bus-0.2.2\src\mutex.rs:191:22
  18: smoltcp_nal::shared::NetworkStackProxy<S>::lock
        at C:\Users\Svoe\.cargo\git\checkouts\smoltcp-nal-f2ec3ae73a86846c\8d2c48e\src\shared.rs:44:9
  19: stabilizer::net::network_processor::NetworkProcessor::update
        at src\net\network_processor.rs:67:15
  20: stabilizer::net::NetworkUsers<S,T>::update
        at src\net\mod.rs:195:37
  21: dual_iir::app::idle::{{closure}}
        at src\bin\dual-iir.rs:395:47
  22: rtic::export::lock
        at C:\Users\Svoe\.cargo\registry\src\github.com-1ecc6299db9ec823\cortex-m-rtic-1.0.0\src\export.rs:152:21
  23: dual_iir::app::<impl rtic_core::Mutex for dual_iir::app::shared_resources::network_that_needs_to_be_locked>::lock
        at src\bin\dual-iir.rs:173:1
  24: dual_iir::app::idle
        at src\bin\dual-iir.rs:395:19
  25: main
        at src\bin\dual-iir.rs:173:1
  26: Reset
(HOST) ERROR the program panicked
error: process didn't exit successfully: `probe-run --chip STM32H743ZITx --speed 30000 'target\thumbv7em-none-eabihf\release\dual-iir'` (exit code: 22)

svrotter avatar Feb 16 '22 16:02 svrotter

The IP I specified was 192.168.137.17

svrotter avatar Feb 16 '22 16:02 svrotter

I just tested current master with 10.34.16.139 and it works fine. Is this --release or --debug?

jordens avatar Feb 16 '22 16:02 jordens

@jordens This only occurs when the STATIC_IP and the BROKER severely mismatch. I had @svrotter update the BROKER_IP to a correct value for his system and the panic went away.

If you do the following, without configuring the BROKER_IP, you should get the panic: STATIC_IP="192.168.137.17" cargo build --release --bin dual-iir

ryan-summers avatar Feb 16 '22 16:02 ryan-summers

Ah. So this may actually be a smoltcp issue. It should say something along the lines of "no route to host". The address is certainly unicast, just unreachable.

jordens avatar Feb 16 '22 16:02 jordens

@jordens I think after Ryans comment it's clearer what this is about but for the sake of completeness: it was the release build

svrotter avatar Feb 16 '22 16:02 svrotter

Interesting. Given the IP 192.168.137.17 I can't see why the assert is_unicast() in lookup() would fail.

jordens avatar Aug 03 '22 16:08 jordens

We got the same panic! when we tried to configure the stabilizer IP on a different subnet than the MQTT broker. For us that's a killer because we have to operate across VLANs.

pmldrmota avatar Aug 31 '22 10:08 pmldrmota

Is there any particular reason you're using STATIC_IP instead of letting DHCP assign an IP? This sounds more like an issue with the network topology instead of an issue with stabilizer

ryan-summers avatar Aug 31 '22 11:08 ryan-summers

The fact that with STATIC_IP there is no gateway support is simply the current state of things and not a bug. @pmldrmota Add it if you are interested. I don't see an easy way to compile-time check for reachability of the broker.

jordens avatar Aug 31 '22 12:08 jordens

Is there any particular reason you're using STATIC_IP instead of letting DHCP assign an IP? This sounds more like an issue with the network topology instead of an issue with stabilizer

IT requires us to register each MAC address with a static IP address (why that is, I don't know). And we ran out of IP addresses on the subnet so we had to overflow into another VLAN.

The fact that with STATIC_IP there is no gateway support is simply the current state of things and not a bug.

@jordens Sorry, I am missing the connection between the panic! and the issue. Is it expected to panic with protocol_addr.is_unicast() if the broker is unreachable? But thank you for pointing out that STATIC_IP does not support gateways, I did not know about this limitation.

pmldrmota avatar Aug 31 '22 12:08 pmldrmota

Registered MACs, static IPs, and DHCP are not at all mutually exclusive. But network management is out of scope here.

As I mentioned above, smoltcp panicing on unroutable (since in this case it's only routable through a gateway) is a bit weird, but yes, that's what it looks like.

jordens avatar Aug 31 '22 13:08 jordens

The broker address is no longer unwrapped, as DNS is used to specify the broker. The BROKER environment variable can be either a valid IP address or a broker specification. Closing as resolved by https://github.com/quartiq/stabilizer/pull/777.

ryan-summers avatar Sep 07 '23 08:09 ryan-summers