network-importer icon indicating copy to clipboard operation
network-importer copied to clipboard

AttributeError: 'NoneType' object has no attribute 'primary_ip'

Open ckozler opened this issue 5 years ago • 11 comments

Environment

  • Python version: 3.7.9
  • Network Importer version: 2.0.0b3

Configuration:

[main]
import_ips = true 
import_prefixes = true
import_cabling = "lldp"       \# Valid options are ["lldp", "cdp", "config", false]
import_intf_status = true     \# If set as False, interface status will be ignore all together
import_vlans = "config"         \# Valid options are ["cli", "config", true, false]
\# excluded_platforms_cabling = ["cisco_asa"]

\# Directory where the configurations can be find, organized in Batfish format
\# configs_directory= "configs"

[inventory]
\# Limit the scope of the inventory to a subset of devices
\# Accept any parameters supported by NetBox devices API
\# filter = "site=nyc,role=router"
\# filter = ""

\# Define a list of supported platform, 
\# if defined all devices without platform or with a different platforms will be removed from the inventory
\# supported_platforms = [ "cisco_ios", "cisco_nxos" ]

[netbox]
\# The information to connect to netbox needs to be provided, either in the config file or as environment variables
address = "http://redacted:8000"                   \# Alternative Env Variable : NETBOX_ADDRESS
token = "redacted"  \# Alternative Env Variable : NETBOX_TOKEN
verify_ssl = false                                   \# Alternative Env Variable : NETBOX_VERIFY_SSL

[network]
\# To be able to pull live information from the devices, the credential information needs to be provided
\# either in the configuration file or as environment variables ( & NETWORK_DEVICE_PWD)
login = "root"      \# Alternative Env Variable : NETWORK_DEVICE_LOGIN
password = "redacted"   \# Alternative Env Variable : NETWORK_DEVICE_PWD

[batfish]
address= "localhost"    \# Alternative Env Variable : BATFISH_ADDRESS
\# api_key = "XXXX"      \# Alternative Env Variable : BATFISH_API_KEY
\# use_ssl = false

[logs]
\# Define log level, curently the logs are printed on the screen
\# level = "info" \# "debug", "info", "warning"
level = "debug"

Steps to Reproduce

  1. Use above configuration file
  2. Run network-importer apply --update-configs or just network-importer apply
  3. Error

Expected Behavior

To complete

Observed Behavior

stack trace

Traceback (most recent call last):
  File "/usr/local/bin/network-importer", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main
    rv = self.invoke(ctx)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "/usr/local/lib/python3.7/site-packages/network_importer/cli.py", line 111, in apply
    ni.sync()
  File "/usr/local/lib/python3.7/site-packages/network_importer/main.py", line 169, in sync
    self.sot.sync_from(self.network, diff_class=NetworkImporterDiff)
  File "/usr/local/lib/python3.7/site-packages/diffsync/__init__.py", line 484, in sync_from
    self._sync_from_diff_element(child, flags=flags, logger=log)
  File "/usr/local/lib/python3.7/site-packages/diffsync/__init__.py", line 577, in _sync_from_diff_element
    self._sync_from_diff_element(child, flags=flags, parent_model=obj, logger=logger)
  File "/usr/local/lib/python3.7/site-packages/diffsync/__init__.py", line 546, in _sync_from_diff_element
    obj = obj.delete()
  File "/usr/local/lib/python3.7/site-packages/network_importer/adapters/netbox_api/models.py", line 248, in delete
    if dev.primary_ip and dev.primary_ip in self.ips:
AttributeError: 'NoneType' object has no attribute 'primary_ip'

ckozler avatar Jan 12 '21 17:01 ckozler

The entire dev object on

https://github.com/networktocode/network-importer/blob/05e5c8eb226343a82382cb9a2a438189ac9555c8/network_importer/adapters/netbox_api/models.py#L255

Is of type 'None'. If I print self I get "ff-ch1-105-ext-sw.redacted.com__irb.4001" which is the device name and irb.4001 is an interface with IP x.x.x.2/24 of which a print of self.ips shows

All IP's and prefixes are imported correctly. This looks like its failing on the very first device and I can also see this in debug log

2021-01-12 12:32:53,315 - network-importer - WARNING - Unable to update interface reth0.4001 on FF_CH_EQ1 in Netbox ({"tagged_vlans":["Invalid pk \"None\" - object does not exist."]})

which comes after the line

2021-01-12 12:32:53,139 - network-importer - DEBUG - Update interface : {'device': 1, 'name': 'reth0.4001', 'type': 'virtual', 'description': 'Trunk LACP to Chicago TOR for internet traffic', 'tagged_vlans': [None], 'lag': None}

The interface is VLAN tagged with tag 4001

ckozler avatar Jan 12 '21 17:01 ckozler

So I guess it was my order of operation which maybe should be documented? I did my very first run with the config you see above however it would not complete until I did each of the [main] section one by one...run 1 with all of them set to false, run 2 just import_ips, run 3 import_ips+import_prefixes true, so on and so forth

After this the run completed without a stack trace

However, I see this in the debug log

2021-01-12 12:39:41,918 - network-importer - DEBUG - Update interface : {'device': 1, 'name': 'reth1.110', 'type': 'virtual', 'description': 'Chicago INFDMZ VLAN', 'tagged_vlans': [None], 'lag': None} 2021-01-12 12:39:42,071 - network-importer - WARNING - Unable to update interface reth1.110 on FF_CH_EQ1 in Netbox ({"tagged_vlans":["Invalid pk \"None\" - object does not exist."]}) 2021-01-12 12:39:42,071 - diffsync - WARNING - 2021-01-12 17:39.42 [warning ] Non-fatal failure encountered [diffsync] action=update diffs={'src': {'mode': 'L3_SUB_VLAN', 'allowed_vlans': ['equinix-chicago-ch1__110']}, 'dst': {'mode': 'NONE', 'allowed_vlans': []}} dst=<Netbox> flags=<DiffSyncFlags.NONE: 0> model=interface src=<Network> status=failure unique_id=FF_CH_EQ1__reth1.110 2021-01-12 12:39:42,071 - diffsync - WARNING - 2021-01-12 17:39.42 [warning ] Not syncing children [diffsync] action=update diffs={'src': {'mode': 'L3_SUB_VLAN', 'allowed_vlans': ['equinix-chicago-ch1__110']}, 'dst': {'mode': 'NONE', 'allowed_vlans': []}} dst=<Netbox> flags=<DiffSyncFlags.NONE: 0> model=interface src=<Network> unique_id=FF_CH_EQ1__reth1.110

Looking at wireshark to look at the request being sent since I see 400's in netbox log and I can see that importer is sending {"tagged_vlans": [null]} in the HTTP request

2021-01-12-124940_1306x722_scrot

ckozler avatar Jan 12 '21 17:01 ckozler

Thanks for the detailed information, it's very useful. I'll try to reproduce the bug locally

The network-importer should be able to work in 1 run, without having to comment out all features one by one

dgarros avatar Jan 13 '21 11:01 dgarros

@dgarros upon adding a new device, it seems I have to do each step before import_vlans will complete. So I need to do import_ips = true with the rest set to false, then I can do import_ips and import_prefixes, and so on and so forth

No special configuration on my devices or anything either

ckozler avatar Jan 22 '21 16:01 ckozler

Actually, disregard, check works but apply does not. New object indicated this time but still dev object null

2021-01-22 11:22:15,120 - network-importer - DEBUG - Found 47 cables from Cli Traceback (most recent call last): File "/usr/local/bin/network-importer", line 8, in sys.exit(main()) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 829, in call return self.main(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 782, in main rv = self.invoke(ctx) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx)) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params) File "/usr/local/lib/python3.7/site-packages/click/core.py", line 610, in invoke return callback(*args, **kwargs) File "/usr/local/lib/python3.7/site-packages/network_importer/cli.py", line 111, in apply ni.sync() File "/usr/local/lib/python3.7/site-packages/network_importer/main.py", line 169, in sync self.sot.sync_from(self.network, diff_class=NetworkImporterDiff) File "/usr/local/lib/python3.7/site-packages/diffsync/init.py", line 484, in sync_from self._sync_from_diff_element(child, flags=flags, logger=log) File "/usr/local/lib/python3.7/site-packages/diffsync/init.py", line 577, in _sync_from_diff_element self._sync_from_diff_element(child, flags=flags, parent_model=obj, logger=logger) File "/usr/local/lib/python3.7/site-packages/diffsync/init.py", line 540, in _sync_from_diff_element obj = obj.update(attrs=diffs["src"]) File "/usr/local/lib/python3.7/site-packages/network_importer/adapters/netbox_api/models.py", line 477, in update nb_params = self.translate_attrs_for_netbox(attrs) File "/usr/local/lib/python3.7/site-packages/network_importer/adapters/netbox_api/models.py", line 433, in translate_attrs_for_netbox if vlan.remote_id: AttributeError: 'NoneType' object has no attribute 'remote_id'

ckozler avatar Jan 22 '21 16:01 ckozler

Actually, I think this comes down to VLAN information not supported in napalm for Juniper

ckozler avatar Jan 22 '21 18:01 ckozler

@ckozler I'm currently running into the same problem. AttributeError: 'NoneType' object has no attribute 'primary_ip' The dev object is None.

Debug Log

2021-02-12 10:29:30,745 - network-importer - INFO - Import SOT Model
2021-02-12 10:29:31,027 - network-importer - DEBUG - Netbox | Found 1 interfaces for DEVICE
2021-02-12 10:29:31,085 - network-importer - DEBUG - Netbox | Found 1 ip addresses for DEVICE
2021-02-12 10:29:31,112 - network-importer - DEBUG - Netbox | Found 0 cables in netbox for site-slug
2021-02-12 10:29:31,112 - network-importer - INFO - Import Network Model
2021-02-12 10:29:32,373 - network-importer - WARNING - Unable to find information for R113-OV in Batfish, SKIPPING

self._data[modelname] - file __init__.py
defaultdict(<class 'dict'>, {'site': {}, 'device': {}, 'interface': {'DEVICE__Loopback0': interface "DEVICE__Loopback0"}, 'ip_address': {'123.456.789.22/32': ip_address "123.456.789.22/32"}, 'cable': {}})
uid- DEVICE

Then the error of NoneType appears

Kani999 avatar Feb 12 '21 09:02 Kani999

I just published a new version in Pypi : 2.0.0-beta4 that is solving this issue.

dgarros avatar Feb 12 '21 12:02 dgarros

I just published a new version in Pypi : 2.0.0-beta4 that is solving this issue. @dgarros Thank you. There is no problem with the error above.


May I have a question? I'm now ending with

raise ObjectNotFound(f"{modelname} {uid} not present in {self.name}")
diffsync.exceptions.ObjectNotFound: device R113-OV not present in Netbox

The device R113-OV is definitely in the netbox. I've imported it by onboarding plugin

Kani999 avatar Feb 12 '21 12:02 Kani999

@Kani999 Please can you open a new issue to track this one and provide your configuration file and the output of the cli. thanks

dgarros avatar Feb 12 '21 12:02 dgarros

Also please can you check if you device is present in network-importer inventory

dgarros avatar Feb 12 '21 12:02 dgarros