Merge pull request #484 from nautobot/release-v2.7.0

Release v2.7.0

README.md

@@ -31,6 +31,7 @@ This Nautobot application framework includes the following integrations:
 - Device42
 - Infoblox
 - IPFabric
+- Itential
 - ServiceNow
 
 Read more about integrations [here](https://docs.nautobot.com/projects/ssot/en/latest/user/integrations). To enable and configure integrations follow the instructions from [the install guide](https://docs.nautobot.com/projects/ssot/en/latest/admin/install/#integrations-configuration).
@@ -79,6 +80,7 @@ The SSoT framework includes a number of integrations with external Systems of Re
 * Arista CloudVision
 * Device42
 * Infoblox
+* Itential
 * ServiceNow
 
 > Note that the Arista CloudVision integration is currently incompatible with the [Arista Labs](https://labs.arista.com/) environment due to a TLS issue. It has been confirmed to work in on-prem environments previously.
@@ -114,6 +116,9 @@ This project includes code originally written in separate Nautobot apps, which h
     [@nniehoff](https://github.com/nniehoff),
     [@qduk](https://github.com/qduk),
     [@ubajze](https://github.com/ubajze)
+- [nautobot-plugin-ssot-device42](https://github.com/nautobot/nautobot-plugin-ssot-device42):
+    Thanks
+    [@jdrew82](https://github.com/jdrew82)
 - [nautobot-plugin-ssot-infoblox](https://github.com/nautobot/nautobot-plugin-ssot-infoblox):
     Thanks
     [@FragmentedPacket](https://github.com/FragmentedPacket),

changes/484.housekeeping

@@ -0,0 +1 @@
+Releasing 2.7.0

development/creds.example.env

@@ -29,6 +29,8 @@ MYSQL_PASSWORD=${NAUTOBOT_DB_PASSWORD}
 NAUTOBOT_ARISTACV_CVP_PASSWORD="changeme"
 NAUTOBOT_ARISTACV_CVP_TOKEN="changeme"
 
+NAUTOBOT_SSOT_DEVICE42_PASSWORD="changeme"
+
 NAUTOBOT_SSOT_INFOBLOX_PASSWORD="changeme"
 
 # ACI Credentials.  Append friendly name to the end to identify each APIC. 

development/development.env

@@ -45,7 +45,7 @@ NAUTOBOT_CELERY_TASK_TIME_LIMIT=7200
 NAUTOBOT_SSOT_HIDE_EXAMPLE_JOBS="False"
 NAUTOBOT_SSOT_ALLOW_CONFLICTING_APPS="False"
 
-NAUTOBOT_SSOT_ENABLE_ACI="True"
+NAUTOBOT_SSOT_ENABLE_ACI="False"
 NAUTOBOT_SSOT_ACI_TAG="ACI"
 NAUTOBOT_SSOT_ACI_TAG_COLOR="0047AB"
 NAUTOBOT_SSOT_ACI_TAG_UP="UP"
@@ -57,7 +57,7 @@ NAUTOBOT_SSOT_ACI_IGNORE_TENANTS="[mgmt,infra]"
 NAUTOBOT_SSOT_ACI_COMMENTS="Created by ACI SSoT Integration"
 NAUTOBOT_SSOT_ACI_SITE="Data Center"
 
-NAUTOBOT_SSOT_ENABLE_ARISTACV="True"
+NAUTOBOT_SSOT_ENABLE_ARISTACV="False"
 NAUTOBOT_ARISTACV_CONTROLLER_SITE=""
 NAUTOBOT_ARISTACV_CREATE_CONTROLLER="True"
 NAUTOBOT_ARISTACV_CVAAS_URL="www.arista.io:443"
@@ -69,12 +69,12 @@ NAUTOBOT_ARISTACV_IMPORT_ACTIVE="False"
 NAUTOBOT_ARISTACV_IMPORT_TAG="False"
 NAUTOBOT_ARISTACV_VERIFY=True
 
-NAUTOBOT_SSOT_ENABLE_DEVICE42="True"
+NAUTOBOT_SSOT_ENABLE_DEVICE42="False"
 NAUTOBOT_SSOT_DEVICE42_HOST=""
 NAUTOBOT_SSOT_DEVICE42_USERNAME=""
 NAUTOBOT_SSOT_DEVICE42_PASSWORD=""
 
-NAUTOBOT_SSOT_ENABLE_INFOBLOX="True"
+NAUTOBOT_SSOT_ENABLE_INFOBLOX="False"
 NAUTOBOT_SSOT_INFOBLOX_DEFAULT_STATUS="Active"
 NAUTOBOT_SSOT_INFOBLOX_ENABLE_SYNC_TO_INFOBLOX="True"
 NAUTOBOT_SSOT_INFOBLOX_IMPORT_OBJECTS_IP_ADDRESSES="True"
@@ -88,11 +88,13 @@ NAUTOBOT_SSOT_INFOBLOX_USERNAME="changeme"
 NAUTOBOT_SSOT_INFOBLOX_VERIFY_SSL="True"
 # NAUTOBOT_SSOT_INFOBLOX_WAPI_VERSION=""
 
-NAUTOBOT_SSOT_ENABLE_SERVICENOW="True"
+NAUTOBOT_SSOT_ENABLE_SERVICENOW="False"
 SERVICENOW_INSTANCE=""
 SERVICENOW_USERNAME=""
 
-NAUTOBOT_SSOT_ENABLE_IPFABRIC="True"
+NAUTOBOT_SSOT_ENABLE_IPFABRIC="False"
 IPFABRIC_HOST="https://ipfabric.example.com"
 IPFABRIC_SSL_VERIFY="True"
 IPFABRIC_TIMEOUT=15
+
+NAUTOBOT_SSOT_ENABLE_ITENTIAL="True"

development/docker-compose.mysql.yml

@@ -19,7 +19,6 @@ services:
   db:
     image: "mysql:8"
     command:
-      - "--default-authentication-plugin=mysql_native_password"
       - "--max_connections=1000"
     env_file:
       - "development.env"

development/nautobot_config.py

@@ -196,6 +196,7 @@
         "enable_device42": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_DEVICE42")),
         "enable_infoblox": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_INFOBLOX")),
         "enable_ipfabric": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_IPFABRIC")),
+        "enable_itential": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_ITENTIAL")),
         "enable_servicenow": is_truthy(os.getenv("NAUTOBOT_SSOT_ENABLE_SERVICENOW")),
         "hide_example_jobs": is_truthy(os.getenv("NAUTOBOT_SSOT_HIDE_EXAMPLE_JOBS")),
         "device42_host": os.getenv("NAUTOBOT_SSOT_DEVICE42_HOST", ""),

docs/admin/integrations/index.md

@@ -7,4 +7,5 @@ This Nautobot app supports the following integrations:
 - [Device42](./device42_setup.md)
 - [Infoblox](./infoblox_setup.md)
 - [IPFabric](./ipfabric_setup.md)
+- [Itential](./itential_setup.md)
 - [ServiceNow](./servicenow_setup.md)

docs/admin/integrations/itential_setup.md

@@ -0,0 +1,63 @@
+# Itential Integration Setup
+
+This guide will walk you through steps to set up Itential integration with the `nautobot_ssot` app.
+
+## Prerequisites
+
+Before configuring the integration, please ensure, that `nautobot-ssot` app was [installed with the Itential integration extra dependencies](../install.md#install-guide).
+
+```shell
+pip install nautobot-ssot[itential]
+```
+
+## Configuration
+
+The integration with Itential primarily utilizes the [External Integrations](https://docs.nautobot.com/projects/core/en/stable/user-guide/platform-functionality/externalintegration/?h=external) and [Secrets](https://docs.nautobot.com/projects/core/en/stable/user-guide/platform-functionality/secret/?h=secrets) features within Nautobot to set up the integration. To enable this integration, the only modification needed is to activate it in the nautobot_config.py file.
+
+Below is an example snippet from `nautobot_config.py` that demonstrates how to enable the Itential integration:
+
+```python
+PLUGINS_CONFIG = {
+    "nautobot_ssot": {
+        "enable_itential": True,
+    }
+}
+```
+
+Remaining configurations are performed in the Nautobot UI or through the Nautobot API.
+
+### Secrets
+
+The Itential integration necessitates four secret values: (1) Itential API access username, (2) Itential API access password, (3) network device access username, and (4) network device access password. You can store these secrets using the secrets provider of your choice.
+
+### Secrets Group
+
+When assigning secrets to a secrets group, please refer to the table below to correctly assign each secret to its respective access type and secret type.
+
+| Secret Description    | Access Type | Secret Type |
+|-----------------------|-------------|-------------|
+| Itential API username | REST        | Username    |
+| Itential API password | REST        | Password    |
+| Device username       | GENERIC     | Username    |
+| Device password       | GENERIC     | Password    |
+
+### External Integration
+
+When setting up an external integration, you must provide the following required fields:
+
+1. **Name**: The unique identifier for the integration.
+2. **Remote URL**: The endpoint URL, including the protocol and port, if applicable.
+3. **Verify SSL**: A boolean value indicating whether SSL certificates should be verified.
+4. **Secrets Group**: The group of secrets associated with the integration, containing necessary authentication details.
+
+The remote URL must include both the protocol (either http or https) and the TCP port used by the automation gateway. For example, to access the automation gateway, you would enter a URL like: https://iag.example.com:8443.
+
+### Automation Gateway Management
+
+To manage the Automation Gateway, navigate to Plugins -> Single Source of Truth -> Itential Automation Gateway in your application. From this interface, you can input details about the automation gateway, which include:
+
+1. **Name**: Specify the name of the automation gateway.
+2. **Description**: Provide a brief description of what the automation gateway is used for.
+3. **Location**: Indicate the primary location of the devices managed by the automation gateway.
+4. **Location Descendants**: This boolean value determines whether the automation gateway should also manage devices in child locations of the specified primary location.
+5. **Enabled**: This boolean setting allows you to enable or disable inventory synchronization with the automation gateway.

docs/admin/integrations/servicenow_setup.md

@@ -80,3 +80,4 @@ PLUGINS_CONFIG = {
 
 !!! note
     Configuration keys are prefixed with `servicenow_`.
+

docs/admin/release_notes/version_2.7.md

@@ -0,0 +1,49 @@
+
+## [v2.7.0 (2024-07-16)](https://github.com/nautobot/nautobot-app-ssot/releases/tag/v2.7.0)
+
+### Added
+
+- [#432](https://github.com/nautobot/nautobot-app-ssot/issues/432) - Added an SSoT to sync Nautobot ==> Itential Automation Gateway.
+- [#432](https://github.com/nautobot/nautobot-app-ssot/issues/432) - This integration allows users to sync Nautobot device inventory to Itential Automation Gateway(s) (IAG).
+- [#432](https://github.com/nautobot/nautobot-app-ssot/issues/432) - The current IAG inventory that is supported is its default Ansible inventory.
+- [#432](https://github.com/nautobot/nautobot-app-ssot/issues/432) - Netmiko, Nornir, HTTP requests inventories will be added at a later date.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Added plugin configuration page collecting configurations for integrations.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added SSOTInfobloxConfig model used for providing Infoblox integration configuration.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added support for multiple configuration instances.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added support for Infoblox Network Views and Nautobot Namespaces.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added support for selecting a subset of Network and IP address objects loaded for synchronization.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added support for creating Infoblox IP Addresses as A and PTR records.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added support for creating Infoblox IP Addresses as Fixed Address records of type RESERVED and MAC_ADDRESS.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added support for excluding extensive attributes and custom fields when synchronizing objects.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added support for selectively enabling synchronization of IPv4 and IPv6 objects.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added support for specifying Infoblox DNS View where DNS records are created.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added support for specifying record types subject to deletion in Infoblox and Nautobot.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - added methods to Infoblox handling fixed addresses, DNS A, Host and PTR records, network views, DNS views, and authoritative zones.
+- [#469](https://github.com/nautobot/nautobot-app-ssot/issues/469) - Added more models for import in Example Jobs.
+
+### Changed
+
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - configuration settings are now defined in the instances of the SSOTInfobloxConfig model.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - functionality provided by the `infoblox_import_subnets` settings has been replaced with the `infoblox_sync_filters` field in the SSOTInfobloxConfig instance.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - updated Infoblox client methods to support Network View.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - standardized `JSONDecoderError` handling in the Infoblox client.
+
+### Removed
+
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - configuration settings defined in `nautobot_config.py` have been removed.
+- [#442](https://github.com/nautobot/nautobot-app-ssot/issues/442) - Infoblox integration - configuration settings defined in environmental variables have been removed.
+
+### Fixed
+
+- [#234](https://github.com/nautobot/nautobot-app-ssot/issues/234) - Fixed integration tests so they're no longer dependent upon being enabled in dev environment.
+- [#437](https://github.com/nautobot/nautobot-app-ssot/issues/437) - Fixed link from list view to filtered sync log view by changing filter query to `sync` from overview.
+- [#443](https://github.com/nautobot/nautobot-app-ssot/issues/443) - Fixed issue with loading duplicate IPAddresses from Infoblox.
+- [#456](https://github.com/nautobot/nautobot-app-ssot/issues/456) - Fix Device42 integration unit test that was expecting wrong BIG-IP netmiko platform name.
+- [#463](https://github.com/nautobot/nautobot-app-ssot/issues/463) - Fixed call in CVP integration to pass `import_active` config setting to get_devices() function call.
+- [#479](https://github.com/nautobot/nautobot-app-ssot/issues/479) - Correct get_or_instantiate() to use self.device_type instead of "device_type" in ACI adapter.
+- [#479](https://github.com/nautobot/nautobot-app-ssot/issues/479) - Refactor load_interfaces() to have check for device_specs var being defined in case file isn't loaded.
+
+### Documentation
+
+- [#450](https://github.com/nautobot/nautobot-app-ssot/issues/450) - Add missing attribution for Device42 integration to README.
+- [#472](https://github.com/nautobot/nautobot-app-ssot/issues/472) - Update ServiceNow documentation for Locations and FAQ error.

docs/user/faq.md

@@ -3,3 +3,11 @@
 ## _Is the application actually a Single Source of Truth?_
 
 In reality the application intends to have behaviors as if it was a SSoT. The difference being, the application intends to aggregate data in the real world where it is not feasible to have the System of Record be in a single system.
+
+## Why did my ServiceNow job fail with an `IncompleteJSONError`?
+
+```
+An exception occurred: `IncompleteJSONError: lexical error: invalid char in json text. <meta (right here) ------^
+```
+
+This exception probably means that your ServiceNow developer instance is currently hibernating and needs to be awoken.

docs/user/integrations/index.md

@@ -7,4 +7,5 @@ This Nautobot app supports the following integrations:
 - [Device42](./device42.md)
 - [Infoblox](./infoblox.md)
 - [IPFabric](./ipfabric.md)
+- [Itential](./itential.md)
 - [ServiceNow](./servicenow.md)

docs/user/integrations/itential.md

@@ -0,0 +1,38 @@
+## Itential Automation Gateway SSoT Integration
+
+The Itential SSoT integration is built as part of the Nautobot Single Source of Truth (SSoT) app. This app enables Nautobot to serve as the aggregation point for inventories in the Itential Automation Gateway (IAG).
+
+## IAG Inventory
+
+The IAG can communicate with network devices using several methods. Each of these communication methods requires a separate inventory in the IAG database. Currently, only the Ansible/Scripts inventory is supported. Support for other inventories may be added at a later date.
+
+Below is a table displaying the IAG communication methods.
+
+| Communication Method |
+| ---------------------|
+| Ansible              |
+| GRPC                 |
+| HTTP Requests        |
+| NETCONF              |
+| Netmiko              |
+| Nornir               |
+| Scripts              |
+| Terraform            |
+
+As a side note, you can create your own scripts or Ansible playbooks that utilize communication methods such as GRPC, HTTP, NETCONF, etc. Itential will use the default Ansible inventory to perform tasks on remote devices with these communication methods. However, if you want the Itential Automation Platform (IAP) to communicate with a remote device using a communication method that is not Ansible or scripts, it requires a separate inventory and iteration by the SSoT App to support those inventories.
+
+## Nautobot to Itential Automation Gateway Modeling
+
+### Device Modeling
+
+Currently, the Itential SSoT integration supports only a one-way sync from Nautobot to the (IAG) devices. For a device object to be synced to an (IAG) host, certain data is required in Nautobot:
+
+1. The device must have an [RFC 1123](https://www.rfc-editor.org/rfc/rfc1123) compliant hostname. The IAG will respond with an HTTP error for non-compliant hostnames.
+2. The device must have a management IP address assigned in Nautobot. This management IP address is used to assign the `ansible_host` variable in the IAG inventory.
+3. For Ansible to determine how to communicate with a remote device, the device needs to be assigned to a platform in Nautobot. The platform must have an appropriate network driver assigned. The Itential SSoT Integration will use this network driver to determine how to assign the `ansible_network_os` Ansible variable in the IAG inventory.
+
+Additional device variables can be assigned to IAG inventories by utilizing Nautobot config contexts. Config contexts will be added to the IAG Ansible inventory as a one-to-one mapping. These config contexts can also be used to override variables for a device, such as the `ansible_network_os`.
+
+### Ansible Default Group Modeling
+
+Ansible uses a default group called `all` to define variables that are common across all devices. More specific groups and devices can override the variables defined in the `all` group. The Itential SSoT integration uses the `all` group to define the `ansible_username` and `ansible_password` variables, which IAG uses to communicate with remote devices. The Itential SSoT integration consumes the device secrets defined in the Itential Setup in the admin section of this documentation.

docs/user/integrations/servicenow.md

@@ -4,7 +4,7 @@ This integration provides the ability to synchronize basic data from Nautobot in
 
 - Nautobot Manufacturer table to ServiceNow Company table
 - Nautobot DeviceType table to ServiceNow Hardware Product Model table
-- Nautobot Region and Site tables to ServiceNow Location table
+- Nautobot Locations tables to ServiceNow Location table
 - Nautobot Device table to ServiceNow IP Switch table
 - Nautobot Interface table to ServiceNow Interface table
 

mkdocs.yml

@@ -132,6 +132,7 @@ nav:
       - Compatibility Matrix: "admin/compatibility_matrix.md"
       - Release Notes:
           - "admin/release_notes/index.md"
+          - v2.7: "admin/release_notes/version_2.7.md"
           - v2.6: "admin/release_notes/version_2.6.md"
           - v2.5: "admin/release_notes/version_2.5.md"
           - v2.4: "admin/release_notes/version_2.4.md"