rConfig - Netbox Device Sync

10 mins V7 Pro

Netbox Device Sync is a feature that allows you to sync devices from Netbox to rConfig. This is useful if you have a large number of devices that you want to import into rConfig. You can sync devices from Netbox to rConfig. The table below shows the Netbox version that has been tested with rConfig on the respective rConfig Professional versions.

Upstream Platform	rConfig Version	Upstream Version	Tested
Netbox	7.2.4	4.0.8	Yes

In this guide we will step through the process of setting up Netbox Device Sync. We will cover the following:

• How it works
• Requirements
• Setting up Netbox
• Setting up rConfig
• Extracting hosts
• Transforming and Mapping
• Staging and Importing
• Syncing devices
• Commands

How it works

We designed the entire device sync process to be as simple as possible, but there are a few steps to follow. The process follows a well known ETL (Extract, Transform, Load) pattern. After we are connected to Netbox, we extract the devices in raw format, then we parse the devices and transform and map them to rConfig format. After this we import the devices into an rConfig devices staging table. Finally you can review the devices and import them into the main rConfig devices table. An administrator will design and setup the process after which everything is validated, and then the process is scheduled to run at regular intervals.

We expect the process to be run at regular intervals to keep the devices in sync with Netbox. The process is designed to be idempotent, so it will not create duplicate devices in rConfig, and the staging tables are cleared after the devices are imported into the main rConfig devices table. Any errors that occur during the process are logged and can be reviewed by the administrator in the rConfig logs.

See high level concepts in the Device Sync Overview document.

Requirements

Before we get started we need to get a few things in place. You will need the following:

• rConfig Professional V7.2.4 or later
• Netbox 4.0.8 or later (earlier API versions may work but have not been tested)
• Administrator level access to Netbox
• Administrator level access to rConfig
• Connectivity between Netbox and rConfig hosts

From a design perspective you will need the following on the Netbox Side:

• A user with read access to the Netbox API
• Tag(s) with the devices you want to sync to rConfig
• The hosts to be sync will need to have the following tags (curly braces to be removed example rconfig::category::1 )
  rconfig::credentialset::{id}, rconfig::category::{id}, rconfig::tag::{id}, rconfig::vendor::{id}, rconfig::model::{string}, rconfig::template::{id}, rconfig::prompt::{string}, rconfig::enableprompt::{string}

On the rConfig side you will need the following:

• A credential set to connect to Netbox
• A category to assign to the devices
• A tag(s) to assign to the devices
• A vendor to assign to the devices
• A model to assign to the devices
• A template to assign to the devices
• A prompt to assign to the devices
• An enable prompt to assign to the devices

Setting up Netbox

The first step in the process is to configure Netbox to allow rConfig to connect to it. This is done by creating an API token Netbox API. This can be done from User Management in the Netbox UI.

Setting up rConfig

The next step is to configure rConfig to connect to Netbox. This is done by creating a credential set in rConfig. The credential set will be used by rConfig to connect to Netbox and extract the devices. The credential set will need to have the following details:

• Integration Name
• Netbox base URL
• Netbox API Token
• Tag(s) to filter

Note

Though rConfig allows for multiple tags, the Netbox API supports multiple tags in an AND condition. This means that if you have multiple tags, the devices will need to have all the tags to be extracted. If you have multiple tags and you want to extract devices that have any of the tags, you will need to run the process multiple times with different tags.

Head over to the Integrations section in rConfig and click configure on the Netbox integration.

On the form for the Netbox integration, fill in the details and click Save. Once saved, you can click Test Connection and Test Credentials to verify that rConfig can connect to Netbox. When testing credentials, a successful test will return the first user in the Netbox system. Not the user you are testing with.

Note: If you have self-signed certificates on your Netbox server, you will need to disable SSL verification in the Netbox integration configuration by setting SSL Verify Host to unchecked.

You can select Tags filter hosts from Netbox to rConfig. Be sure to click Add tag after making changes. You can add multiple tags to sync.

Extracting hosts

The first step in the process is to extract the devices from Netbox. This is done using the Netbox API. The devices are extracted in raw format and then parsed and transformed into rConfig format. We expect to apply filters per the Netbox system to only extract the devices that you want to import into rConfig. Once the devices are extracted, they are ready to be imported into rConfig.

Click Next on the Netbox integration configuration screen or go directly to the Extract List tab to extract the devices from Netbox.

In the screenshot below, you can see an empty extract list. We have a three step process to extract, transform and import the devices. The first step is to extract the devices from Netbox. Click the Run extract from Netbox button to start the process. This will take a moment to complete.

Once the extract is completed, you will see a host list similar to the below. You can see the devices that have been extracted from Netbox. You can open devices directly in Netbox by clicking the View in Netbox button. You can also manually delete devices from the list if you do not want to import them into rConfig. But be careful, as as an automated process will re-import the devices on the next run.

Transforming and Mapping

The next step is to transform and map the devices to rConfig format. This is done using tags that need to be applied on the Netbox system. The tags are then used to map the devices in rConfig. The tags are listed above and are used to map the correct fields and values for the final list of imported devices in rConfig. All tags are required for a successful import. Click the Run Transform & Mapping button to start the process.

Once this is completed, you will see some changes in the table. You will notice each row can now be expanded to show the tags that have been applied to the devices. You will also see an indication of whether the devices are ready to be imported into rConfig. If there are any errors, you will see a message in the table. You can click the Status & Logs tab to see the details of any errors.

Staging and Importing

If you have an extracted and transformed list you are happy with, go ahead and click the Load to Device Staging Table button to import the devices into rConfig. Wait for a moment for the process to complete. Any invalid hosts, just will not be loaded. All valid hosts will be loaded into the staging table.

When the staging table is populated, you can review the devices and import them into the main rConfig devices table per the screenshot below. A few things to note on this screen:

• You can see the devices that are in the staging table
• You can expand each row to see the transformation i.e. you have valid rConfig properties for each device based on the mapped tags
• Any devices in this list are ready to be imported

You will also notice two buttons at the top of the table. The Run assessment button will simply run each part of the process in sequence up to the point of loading hosts in to the staging table. This is useful for testing the process before you actually import the devices. The Run full deployment button will run each part of the process in sequence and then import the devices into the main rConfig devices table. This is the final step in the process. When the full deplopyment is complete, the staging table will be cleared.

You should continually monitor the logs for any errors that may occur during the process. You can do this by clicking the Status & Logs tab.

Note the newlly imported devices in the main rConfig devices table. Take note of the indictor icon next to the hostname in the table. This will indicate the source of the device. In this case, the devices have been imported from a device sync integration. The main device view also has this indicator and will have a View in Netbox button on the main device details menu.

Syncing devices

Once the entire process is complete, you can then use the devices in rConfig as you would with any other device. You can run commands, backup configs, and more. The devices will be kept in sync with Netbox as the process is designed to be run at regular intervals. The process is designed to be idempotent, so it will not create duplicate devices in rConfig and the staging tables are cleared after the devices are imported into the main rConfig devices table. Any errors that occur during the process are logged and can be reviewed by the administrator in the rConfig logs.

Syncing devices is as simple as create a scheduled task to run the process at regular intervals. You can do this from the Scheduled Tasks section in rConfig. Click Add Task and select Integration Job from the task type dropdown. On Step 2 set the task name and description.

On step 3, you can select the specific Netbox integration you setup on the Configuration Tab. Step 4 is where you set you schedule. And Step 5 is where you review and save the task.

Commands

As with anything in rConfig, most of the tasks can be run on the CLI for automation, advanced troubleshooting and more. The commands for the Netbox integration are as follows:

php artisan rconfig:integration-netbox

When you run the command without any options, you will see a list of available options. You can run the command with the --help option to see the available options.

Option	Description	shortcut
—connection	Test the connection to Netbox. Assumed that it has been setup in the UI	-c
—usertest	Test the Netbox API user credentials. Assumed that it has been setup in the UI	-u
—extract	Extract the devices from Netbox	-e
—transform	Transform and map the devices to rConfig format	-t
—mapping	Run the mapping process	-m
—load	Load the devices into the staging table	-l
—assessment	Run the assessment process, which runs all previous steps but does not deploy hosts to the main devices table	-a
—deployment	Run the full deployment process, which runs all previous steps and deploys hosts to the main devices table	-d

The various switches are exactly what is run when you click the buttons in the UI. You will get more details in terms of output and errors when you run the command on the CLI.

Conclusion

Netbox Device Sync is a powerful feature that allows you to sync devices from Netbox to rConfig. This is useful if you have a large number of devices that you want to import into rConfig. You can sync devices from Netbox to rConfig. The process is designed to be idempotent, so it will not create duplicate devices in rConfig. and the staging tables are cleared after the devices are imported into the main rConfig devices table. Any errors that occur during the process are logged and can be reviewed by the administrator in the rConfig logs.