Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error creating manufacturers - netbox v3.7.1 #123

Open
howardtopher opened this issue Feb 8, 2024 · 9 comments
Open

Error creating manufacturers - netbox v3.7.1 #123

howardtopher opened this issue Feb 8, 2024 · 9 comments

Comments

@howardtopher
Copy link

howardtopher commented Feb 8, 2024

My guess here is that this script doesn't support netbox v3. But this is the error I get when trying to run the script to do an initial import on a new system. There are currently 0 manufacturers existing on this system yet.

Error creating manufacturers
Error during manufacturer creation. - {"error": "duplicate key value violates unique constraint \"dcim_manufacturer_slug_key\"\nDETAIL:  Key (slug)=(watchguard) already exists.", "exception": "IntegrityError", "netbox_version": "3.7.1", "python_version": "3.9.18"}

I tried changing requirements.txt to bump pynetbox up to 7.3.3 and I still get the same error.

@cbka
Copy link

cbka commented Feb 27, 2024

+1

@howardtopher
Copy link
Author

I did determine that the import works if you specify a vendor to install. I only encounter the error when running the script to install everything at once.

@Konraden
Copy link

The device library itself has "Watchguard" and "WatchGuard" directories containing devices. I'm going guess it breaks when it tires to create a slug the second time. The index is likely case insensitive but the script isn't.

@gurubert
Copy link

gurubert commented Mar 4, 2024

netbox-community/devicetype-library#1978 fixes this issue.

@netravnen
Copy link

netravnen commented Mar 12, 2024

Used the master branch of the repository here for the importer. Tried limiting the vendors to different ones. Without any real success.

# /opt/Device-Type-Library-Import/venv/bin/python3 /opt/Device-Type-Library-Import/nb-dt-import.py --vendors Generic 
Package devicetype-library is already installed, updating /opt/Device-Type-Library-Import/repo
Traceback (most recent call last):
  File "/opt/Device-Type-Library-Import/nb-dt-import.py", line 54, in <module>
    main()
  File "/opt/Device-Type-Library-Import/nb-dt-import.py", line 17, in main
    netbox = NetBox(settings)
             ^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 29, in __init__
    self.existing_manufacturers = self.get_manufacturers()
                                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in get_manufacturers
    return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in <dictcomp>
    return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/response.py", line 127, in __next__
    next(self.response), self.endpoint.api, self.endpoint
    ^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 280, in get
    req = self._make_call(add_params=add_params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 247, in _make_call
    raise RequestError(req)
pynetbox.core.query.RequestError: The request failed with code 500 Internal Server Error but more specific details were not returned in json. Check the NetBox Logs or investigate this exception's error attribute.
# ./nb-dt-import.py --vendors apc
Package devicetype-library is already installed, updating /opt/Device-Type-Library-Import/repo
Traceback (most recent call last):
  File "/opt/Device-Type-Library-Import/./nb-dt-import.py", line 54, in <module>
    main()
  File "/opt/Device-Type-Library-Import/./nb-dt-import.py", line 17, in main
    netbox = NetBox(settings)
             ^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 29, in __init__
    self.existing_manufacturers = self.get_manufacturers()
                                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in get_manufacturers
    return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/netbox_api.py", line 58, in <dictcomp>
    return {str(item): item for item in self.netbox.dcim.manufacturers.all()}
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/response.py", line 127, in __next__
    next(self.response), self.endpoint.api, self.endpoint
    ^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 280, in get
    req = self._make_call(add_params=add_params)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/Device-Type-Library-Import/venv/lib/python3.11/site-packages/pynetbox/core/query.py", line 247, in _make_call
    raise RequestError(req)
pynetbox.core.query.RequestError: The request failed with code 500 Internal Server Error but more specific details were not returned in json. Check the NetBox Logs or investigate this exception's error attribute.

@danner26
Copy link
Member

v3 is supported, but a newer version must have broke this functionality. We will look into it ASAP, or anyone is welcome to submit a PR if the issue is found

@gurubert
Copy link

The PR is already merged: netbox-community/devicetype-library#1978

@netravnen
Copy link

netravnen commented Mar 20, 2024

v3 is supported, but a newer version must have broke this functionality. We will look into it ASAP, or anyone is welcome to submit a PR if the issue is found

Now double-checking, after your response, and ensuring I have pulled the latest import and repo commits. It works... To a degree.

Now I am hitting different kind of errors during the device type import.

  1. conversion between UTF8 and SQL_ASCII is not supported ⟵ Unsure how to got about this one. Installed the netbox in an LXC container according to the manual. Debian 15.6-0+deb12u1.
    1. Postgresql defaults to encoding SQL_ASCII if not specified otherwise.
    2. Followed this blog post.
    3. Stopped netbox.
    4. Took a database-dump of database netbox.
    5. Dropped and recreated template1 to new default encoding (UTF8).
    6. Created new database, netbox1.
    7. Restored database-dump to netbox1.
    8. Updated the configuration.py to point to database netbox1.
    9. Started netbox.
    10. Rerun device importer.
    11. No more conversion errors.
  2. power_port":["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: PS1"Seems easy to resolve. As the device type needs to be updated Stuck on what device is affected here
  3. Error {"u_height":["Child device types must be 0U."]} creating device type: Mellanox SX6012T-2BFS ⟵ Add u_height value to yaml definition. Fix missing u_height for Mellanox/SX6012* devicetype-library#2014
  4. Could not find Rear Port for Front Port: Port/18 - st - 1594 Error '[{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{"rear_port":["Related objects must be referenced by numeric ID or by dictionary of attributes. Received an unrecognized value: Port/18"]},{},{},{},{},{},{}]' creating Front Port ⟵ ... 🤔 💭 Incorrect rear_port ST-24-port-fiber-patch-panel-rear-splice devicetype-library#2015

@mhdan
Copy link

mhdan commented Oct 26, 2024

I did determine that the import works if you specify a vendor to install. I only encounter the error when running the script to install everything at once.

having the same problem, the netbox-community/devicetype-library#1978 PR don't fix this issue and I encounter the issue when trying to import all devices but it's working when trying each vendor standalone.

here is verbose output of error in my case

(venv) root@netbox:/opt/Device-Type-Library-Import# ./nb-dt-import.py --verbose                                                                                                                  [608/1806]Package devicetype-library is already installed, updating /opt/Device-Type-Library-Import/repo                                                                                                             
Pulled Repo https://github.com/netbox-community/devicetype-library.git                                                                                                                                     Netbox version 4.1 found. Using new filters.                                                                                                                                                               
143 Vendors Found                                                                                                                                                                                          3534 Device-Types Found                                                                                                                                                                                    
Manufacturer Exists: AFL - 11                                                                                                                                                                              
Manufacturer queued for addition: Check Point                                                                                                                                                              
Manufacturer queued for addition: Rockwell Automation                                                                                                                                                      Manufacturer queued for addition: Supermicro                                                                                                                                                               
Manufacturer queued for addition: QNAP                                                                                                                                                                     Manufacturer queued for addition: Nvidia                                                                                                                                                                   
Manufacturer queued for addition: Panduit                                                                                                                                                                  Manufacturer queued for addition: D-Link                                                                                                                                                                   
Manufacturer queued for addition: CryptoPro                                                                                                                                                                Manufacturer queued for addition: Delta                                                                                                                                                                    
Manufacturer queued for addition: Moxa                                                                                                                                                                     Manufacturer queued for addition: ZTE                                                                                                                                                                      
Manufacturer Exists: Arista - 2                                                                                                                                                                            Manufacturer queued for addition: ActionTec                                                                                                                                                                
Manufacturer queued for addition: NetApp                                                                                                                                                                   
Manufacturer queued for addition: TrendNet                                                                                                                                                                 Manufacturer queued for addition: Alpha                                                                                                                                                                    
Manufacturer queued for addition: Commscope                                                                                                                                                                Manufacturer queued for addition: Schneider Electric                                                                                                                                                       
Manufacturer queued for addition: Diamond SA                                                                                                                                                               Manufacturer queued for addition: Fujitsu                                                                                                                                                                  
Manufacturer queued for addition: Palo Alto                                                                                                                                                                Manufacturer queued for addition: Avocent                                                                                                                                                                  
Manufacturer queued for addition: Peplink                                                                                                                                                                  Manufacturer queued for addition: Canon                                                                                                                                                                    
Manufacturer queued for addition: Factor-TS                                                                                                                                                                Manufacturer queued for addition: LANCOM                                                                                                                                                                   
Manufacturer queued for addition: Datto                                                                                                                                                                    
Manufacturer queued for addition: Digi                                                                                                                                                                     
Manufacturer queued for addition: Colable                                                                                                                                                                  
Manufacturer queued for addition: WatchGuard                                                                                                                                                               
Manufacturer queued for addition: Opengear                                                                                                                                                                 
Manufacturer queued for addition: Adtran
Manufacturer queued for addition: Qotom
Manufacturer queued for addition: Digitus
Manufacturer queued for addition: Infoblox
Manufacturer queued for addition: QCT
Manufacturer queued for addition: Citrix
Manufacturer queued for addition: CE6885-48YS8CQ.yaml
Manufacturer queued for addition: Sophos
Manufacturer queued for addition: WTI
Manufacturer queued for addition: Calient
Manufacturer queued for addition: Inspur
Manufacturer queued for addition: QTECH
Manufacturer queued for addition: Server Technology
Manufacturer queued for addition: Pluribus
Manufacturer queued for addition: Nozomi Networks
Manufacturer queued for addition: Arctic-Wolf
Manufacturer queued for addition: Brocade
Manufacturer queued for addition: Infinera
Manufacturer queued for addition: Netgear
Manufacturer queued for addition: Ippon
Manufacturer queued for addition: F5
Manufacturer queued for addition: Netonix
Manufacturer queued for addition: Ciena
Manufacturer Exists: Juniper - 1
Manufacturer queued for addition: Eaton
Manufacturer Exists: Cisco - 8
Manufacturer queued for addition: Ericsson
Manufacturer queued for addition: Apple
Manufacturer queued for addition: Thales
Manufacturer queued for addition: Riverbed
Manufacturer queued for addition: Ruckus
Manufacturer queued for addition: Solid Optics
Manufacturer queued for addition: Deciso
Manufacturer queued for addition: ADVA
Manufacturer queued for addition: Datwyler
Manufacturer queued for addition: Mimosa Networks
Manufacturer queued for addition: Intel
Manufacturer queued for addition: Huawei
Manufacturer queued for addition: Hikvision
Manufacturer queued for addition: Leviton
Manufacturer Exists: Dell - 9
Manufacturer queued for addition: ZPE
Manufacturer queued for addition: SonicWall
Manufacturer queued for addition: Corning
Manufacturer queued for addition: CyberPower
Manufacturer queued for addition: Fortinet
Manufacturer queued for addition: Riello
Manufacturer queued for addition: Zonit
Manufacturer queued for addition: Rohde & Schwarz
Manufacturer queued for addition: nexconec
Manufacturer queued for addition: Datacom
Manufacturer queued for addition: A10
Manufacturer queued for addition: Middle Atlantic
Manufacturer queued for addition: Infotecs
Manufacturer queued for addition: DataDirect Networks
Manufacturer queued for addition: Netgate
Manufacturer queued for addition: Generic
Manufacturer queued for addition: TP-Link
Manufacturer queued for addition: LiteOn
Manufacturer queued for addition: MikroTik
Manufacturer queued for addition: Lenovo
Manufacturer queued for addition: Cuda
Manufacturer queued for addition: RAD
Manufacturer queued for addition: APC
Manufacturer queued for addition: Parks
Manufacturer queued for addition: Mellanox
Manufacturer queued for addition: EnGenius
Manufacturer queued for addition: FS
Manufacturer queued for addition: Albis-Elcon
Manufacturer queued for addition: IBM
Manufacturer queued for addition: Nokia
Manufacturer queued for addition: Zyxel
Manufacturer queued for addition: ghipsystems
Manufacturer queued for addition: Overland-Tandberg
Manufacturer queued for addition: Synology
Manufacturer queued for addition: Eltex
Manufacturer queued for addition: Blackmagicdesign 
Manufacturer queued for addition: Barracuda
Manufacturer queued for addition: Raspberry Pi
Manufacturer Exists: HPE - 10
Manufacturer queued for addition: PC Engines
Manufacturer queued for addition: Alcatel-Lucent
Manufacturer queued for addition: Reichle & De-Massari
Manufacturer queued for addition: Aten
Manufacturer queued for addition: Bachmann
Manufacturer queued for addition: YAMAHA
Manufacturer queued for addition: SNR
Manufacturer queued for addition: Rittal
Manufacturer queued for addition: Sedna
Manufacturer queued for addition: DCN
Manufacturer queued for addition: Edgecore
Manufacturer queued for addition: Allied Telesis
Manufacturer queued for addition: Neousys
Manufacturer queued for addition: Teltonika
Manufacturer queued for addition: Digital Loggers
Manufacturer queued for addition: ExaGrid
Manufacturer queued for addition: Brennenstuhl
Manufacturer queued for addition: AVM
Manufacturer queued for addition: Calix
Manufacturer queued for addition: NEC
Manufacturer queued for addition: 3Com
Manufacturer queued for addition: Ubiquiti
Manufacturer queued for addition: V-Solution
Manufacturer queued for addition: Cradlepoint
Manufacturer queued for addition: Ivanti
Manufacturer queued for addition: Extreme Networks 
Manufacturer queued for addition: T-bao
Manufacturer queued for addition: Certa
Manufacturer queued for addition: FireBrick
Manufacturer queued for addition: Raritan
Manufacturer queued for addition: Vertiv
Error creating manufacturers
Error during manufacturer creation. - [{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{}
,{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{"slug":["manufacturer with this slug already exists."]},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{
},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{},{}]
Device Type Exists: AFL - U-Series Housing 2RU, 12 Cassettes - 1652
Device Type Exists: AFL - U-Series Housing 1RU, 5 Cassettes - 1653
Error {"manufacturer":["Related object not found using the provided attributes: {'name': 'Check Point', 'slug': 'check-point'}"]} creating device type: Check Point SG6400
Error {"manufacturer":["Related object not found using the provided attributes: {'name': 'Check Point', 'slug': 'check-point'}"]} creating device type: Check Point SG23800
Error {"manufacturer":["Related object not found using the provided attributes: {'name': 'Check Point', 'slug': 'check-point'}"]} creating device type: Check Point SG3600
Error {"manufacturer":["Related object not found using the provided attributes: {'name': 'Check Point', 'slug': 'check-point'}"]} creating device type: Check Point CPAC-1500/3600/3800-RM-DUAL

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants