rConfig - xFTP Server

5 mins V7 Pro

Since version 6.6.0, rConfig has included built-in TFTP services. These services have been useful for uploading non-text or configuration content to the rConfig server for backup and archival purposes. However, TFTP itself is an insecure and UDP-based protocol.

With the release of rConfig v7.1.0, FTP and SFTP services have been introduced to fulfill the same use cases while utilizing more secure and reliable technologies. This enhancement ensures the integrity and security of data transfers, aligning with best practices for secure communications.

Breaking Change

If you were utilizing the previous rConfig TFTP service, please note that this is a breaking change. You will need to remove the existing TFTP service and set up the new Docker-based TFTP service introduced in rConfig v7.1.0. Detailed instructions for the removal of the old service and the setup of the new Docker-based TFTP service can be found in the guide below. This transition is necessary to ensure compatibility and leverage the enhanced security features provided in this version.

Microservices Architecture Transition

rConfig is transitioning towards a microservices-based architecture for services such as TFTP, FTP, SFTP, and other future services. Docker will be utilized as the container deployment tool for these microservices.

The setup of these various microservices has been simplified and can be easily managed through the rConfig UI and CLI-based tools. This architectural shift aims to enhance modularity, scalability, and ease of maintenance for rConfig services.

Microservices File Management

The various microservices in rConfig serve as an abstracted frontend for uploaded files via the different xFTP protocols. During installation, you will set the credentials for FTP and SFTP. Note that TFTP does not require credentials. These credentials are stored in the .env file for future use and retrieval.

Each microservice’s upload directory is mapped to a specific directory on the rConfig server (on all supported operating systems). This mapping ensures that all uploaded files are organized and accessible within the server’s file system. Detailed information on directory mapping and credential management can be found in the installation and configuration documentation.

/srv/xftp/

As with the legacy TFTP service, the upload directory for each microservice is expected to remain empty at all times. This is because rConfig automatically processes uploaded files containing a specific DeviceId in their names. Once detected, rConfig moves these files to the respective device’s FTP folder on the disk.

To run this file processing operation, use the following command:

php artisan rconfig:sweep-tftp-dir

This command can be executed manually at any time and is set to run automatically every 15 minutes by default. The frequency of this task can be adjusted in the ‘Scheduled Tasks’ settings after the task has been deployed.

Ephemeral Nature of Microservices

The microservices in rConfig are designed to be ephemeral, meaning they are short-lived and can be dynamically created and destroyed as needed. This design allows for rapid scaling, flexibility in deployment, and resilience in handling failures. Services can be quickly replaced or updated without impacting the overall system, ensuring continuous and efficient operation.

xFTP Services

We need to get docker installed before we can start setting up the xFTP microservices.

From V7.1.0, rConfig supports TFTP, FTP and SFTP microservices with a view to support the following use cases;

1. Uploading non text/ binary files from network devices to the rConfig server
2. Uploading tar or other archive type files from network devices to the rConfig server
3. Storing those uploaded files with the relevant device ID so that they may be time stamped/ version and easily accessible via the given devices page in the rConfig UI.

Feature Update

The TFTP Feature was located in Settings > TFTP Server, but this page has been rebranded as Settings > xFTP Services. NOTICE: Presently offline rConfig installations may require extra support to setup Docker and download the various images.

You can verify the status of Docker and the microservices by opening the Settings > xFTP Services page. It will look like the below screenshot by default.

Setting up Docker

We need to get docker installed before we can start setting up the xFTP microservices.

How to install Docker

We are pleased to introduce the php artisan rconfig:xftp-toggle command in rConfig. This command facilitates the setup of Docker and the associated microservices. By running this command, most tasks related to configuring Docker and microservices will be automated.

To install Docker, execute the following command from the server CLI:

cd /var/www/html/rconfig7/current && php artisan rconfig:xftp-toggle

You will be guided through a set of questions to complete the setup process. This interactive setup ensures that all necessary configurations are correctly applied, streamlining the installation and deployment of Docker and the microservices.

Select ‘yes’ and Docker will install. This may take a minute or two, and will depend on your access to the internet and Docker repositories. You may, if you wish install Docker manually for your given OS, and once Docker -v command returns a correct version, the rConfig microservice installation can continue.

Once the installation is complete, you will then be asked which microservice you wish to install. Select your preferred service to continue.

Install multiple services

Each of the xFTP services in rConfig can be run in parallel. This capability allows multiple file transfer protocols (TFTP, FTP, SFTP) to operate simultaneously, enhancing the system’s efficiency and flexibility. The parallel execution ensures that different types of file transfer operations can be handled concurrently, optimizing performance and reducing potential bottlenecks in data transfer processes.

Back on the xFTP Settings page the UI, refresh the page and the Docker Status should change to ‘Running’, and the Docker version number should be present per the screenshot below.

Removing Docker

We have provided some convenient scripts below for removing docker depending on your OS. Please refer to your specific OS documentation for more information on removing Docker.

How to remove docker for Rocky/RHEL/CentOS 9

sudo dnf remove docker-ce docker-ce-cli containerd.io docker-compose-plugin -y
sudo rm -rf /var/lib/docker
sudo rm -rf /var/lib/containerd

How to remove docker for Ubuntu

sudo apt-get purge docker-ce docker-ce-cli containerd.io docker-compose-plugin
sudo rm -rf /var/lib/docker
sudo rm -rf /var/lib/containerd

xFTP Status

You may view the status of Docker and the various xFTP microservices from the Settings > xFtp Services UI page, or from the CLI. Use the CLI command below to view the status of Docker and the microservices.

cd /var/www/html/rconfig7/current && php artisan rconfig:xftp-toggle --status

Running Commands

All examples below assume you are running the php artisan command from within the /var/www/html/rconfig7/current

cd /var/www/html/rconfig7/current

Setting up XFTP Services

Choose from one of the tabs below for the service you wish to setup.

TFTP Services

Setting up TFTP Services

Downloading Docker Images

If this is your first time setting up any of the microservices, it may take a minute to download the images from dockerhub. Please be patient.

Install the TFTP microservice is as simple as running the command below and following the prompts.

cd /var/www/html/rconfig7/current && php artisan rconfig:xftp-toggle

Stopping TFTP Services

You may also stop/ remove the service by running the same command again.

FTP Services

Setting up FTP Services

Downloading Docker Images

If this is your first time setting up any of the microservices, it may take a minute to download the images from dockerhub. Please be patient.

Setting up FTP Services

Setting up the FTP service is the same as setting up TFTP, except this time we need to add a set of credentials.

Stopping FTP Services

You may also stop/ remove the service by running the same command again.

FTP Credentials

When you start the FTP service a second time, you will note that you are asked if you wish to reuse the previously create credentials. This allows you to a) update the previously created credentials, and b) recreate the microservice without having to re-specify the initially created credentials.

SFTP/SCP Services

Setting up SFTP/SCP Services

Downloading Docker Images

If this is your first time setting up any of the microservices, it may take a minute to download the images from dockerhub. Please be patient.

Setting up the SFTP service is the same as setting up TFTP, except this time we need to add a set of credentials.

SFTP vs SCP

You can use the SFTP Service to service SCP connections, but there are differences between the protocols.

Service Port Number

As SFTP/ SCP rely on the underlying SSH protocol, which defaults to port 22 on hosts - The microservice that runs the SFTP service will need a different port number so that it does not conflict with the hosts SSH daemon port (usually 22). You will be prompted for a different port number to 22 during the setup.

Setting up SFTP/SCP Services

See steps below to add the SFTP service, and set the credentials and port number.

Stopping SFTP/SCP Services

You may also stop/ remove the service by running the same command again.

FTP Credentials

When you start the FTP service a second time, you will note that you are asked if you wish to reuse the previously create credentials. This allows you to a) update the previously created credentials, and b) recreate the microservice without having to re-specify the initially created credentials.

====================================================================================================================================================

How it works

Once the xFTP service of your choice is enabled, the service will be available on the rConfig server. The service will be listening on the default ports for the given protocol. The service will be accessible from the rConfig server itself and is not accessible from the internet, unless you open a port to it from the internet. Which would be a bad idea!

You can then copy files from your network devices to the rConfig server using the xFTP service. The files will be stored in the /srv/xftp/ directory on the rConfig server. The files will be stored with the device ID in the filename. The device ID is a unique number assigned to each device in rConfig. The device ID is not the same as the device name. The device ID is a number that is assigned to each device in the order that the device was added to rConfig. The device ID is visible in the GUI. The device ID is also visible in the database in the devices table in the id column.

Strictly speaking, your xFTP directory /srv/xftp/ should be empty at all times, as rConfig sweeps the directory per the deployed scheduled task TFTP Directory Sweep and should pick up files to place them with the correct network device - If a device id is present in the filename i.e. vlan-1234.dat. You may manually delete files from your TFTP directory if you wish.

TFTP Sweep task automatically deployed with TFTP Setup. Frequency of runs can be adjusted.

The sweep task looks at the filenames in the directory, and if any filenames contain the ID of a device, it moves that file to the devices storage location. The file will then be presented on the Device View page. You may also sweep the directory manually using the following command:

cd /var/www/html/rconfig7/current && php artisan rconfig:sweep-tftp-dir
Found 1 valid file in TFTP directory.

If any files are found without a valid device ID in the filename they will not be moved and will remain in the TFTP directory. You may manually remove these files from the xFTP settings page.

How to upload files manually

For simplicty we will use a TFTP example. To copy device files from the devices local storage to the rConfig TFTP server for processing and storage you must use the copy tftp command. The copy tftp command is a Cisco IOS command and is used to copy files from the device to a TFTP server. The copy tftp command is not supported on all devices. You should check your own devices documentation to see if it supports the copy tftp or similar command.

When adding the copy tftp command to a Cisco device, you must specify the TFTP server IP address and the filename to be copied. The filename must be unique and must not contain any spaces. The filename must also be unique across all devices.

The following example shows how to copy a Cisco IOS device configuration file to the rConfig TFTP server:

Router#conf t
Router(config)#file prompt quiet
router1#copy csrlxc-cfg.log tftp://192.168.1.131/csrlxc-cfg-20003.log
!!
376 bytes copied in 0.004 secs (94000 bytes/sec)

You will need the correct copy commands for FTP and SFTP services for your network devices. Please refer to your devices documentation for the correct commands.

Secure credentials for commands

You have the ability to keep FTP and SFTP credentials secure in the UI. As theses are hardcoded to the .env file when you configure the FTP/SFTP services, they can be retrieved automatically with certain command parameters. See examples below.

copy cpconfig-19xx.cfg ftp://{ftpusername}:{ftppassword}@192.168.1.6/{deviceid}-cpconfig-19xx.cfg

copy cpconfig-19xx.cfg scp://{sftpusername}:{sftppassword}@192.168.1.6/{deviceid}-cpconfig-19xx.cfg

Please notice the following parameters;

• {ftpusername} - The FTP username
• {ftppassword} - The FTP password
• {sftpusername} - The SFTP username
• {sftppassword} - The SFTP password

When the command is run, these are replaced by the values as seting in the .env file. The .env file values are set when you configure the FTP/SFTP services in the CLI. You may use all of the above examples and parameters in the rConfig commands section.

Please note the use of ‘file prompt quiet’ to suppress the prompt for the filename. This is not required but is a good practice to avoid the prompt on Cisco devices. Check with other vendors documentation to see if this is supported.

See output of TFTP directory in the GUI below after the file was uploaded successfully.

How to upload files automatically

We can automate the uploaded of such files from the local storage of Network devices (vlan.dat files etc..) by add the copy command as part of the devices command/ category set in rConfig. This will allow the file to be uploaded to the TFTP server automatically when the device is polled. This is useful for files that are not updated often and are not likely to change. See an example below as configured in the commands section of the rConfig GUI.

copy csrlxc-cfg.log tftp://192.168.1.131/{deviceid}-csrlxc-cfg.log

Take special note of the variable entered in the command. This is the device ID of the device. This is a variable that is replaced with the actual device ID when the command is run. This is how rConfig knows which device the file belongs to. The device ID is a unique number assigned to each device in rConfig. The device ID is not the same as the device name. The device ID is a number that is assigned to each device in the order that the device was added to rConfig. The device ID is visible in the GUI. The device ID is also visible in the database in the devices table in the id column. Also, a timestamp will be appended to the filename once saved to the TFTP directory. This is to ensure that the file is unique and will not be overwritten by a subsequent upload.

Credential Protection for FTP and SFTP

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

How to view uploaded files

If files have the correct device ID in the filename, they will be moved to the devices storage location and will be presented on the Device View page. Go to the device view page and a new table will be available at the bottom of the page.

Click View all to view all uploaded files, and for more actions, such as view, delete and download.