Back up things that are not network devices
rConfig doesn’t care where the text comes from. If a script prints it and exits cleanly, it’s versioned, diffed and searchable. That makes SIE a simple way to watch the configuration of things around the network: the database an application depends on, the Linux servers that run your DNS or RADIUS, the firewall rules in a cloud account. This page has three short recipes, one for each.
All three use the same template, with one marker each:
main: name: "SIE - Not a network device" desc: "Versions the configuration of a database, server or cloud account"
connect: timeout: 120 idletimeout: 60 idleTimeout: 60 protocol: script
# exit_codes is deliberately left out: any non-zero exit fails the run.failure_criteria: error_patterns: - "ERROR:" success_patterns: - "SIE-OK:"Give each system its own rConfig device record. The IP address is the system’s address, and the name is whatever your team calls it.
A database schema
Stores the structure of a MySQL or MariaDB database (tables, columns, indexes, views and stored procedures) without any of its data. A dropped index or a changed column shows up the same way a changed ACL does on a switch.
The database login goes in an option file in the scripts keys folder, owned by the web user and mode 0600, so the password never appears on a command line:
[client]user=schema_readerpassword=example-passwordThe account only needs to read the schema, never to change it. The exact privileges for dumping views, routines, triggers and events differ between MySQL and MariaDB versions, so check the mysqldump documentation for yours.
Command (the database name is the last word; change it per device):
/bin/bash {script_path}db-schema.sh {device_ip} appdb#!/usr/bin/env bash## Store a MySQL or MariaDB schema, without data, for the rConfig Script# Integration Engine.## Usage: db-schema.sh <db_host> <database>
set -o pipefail
OPTION_FILE="/var/www/html/rconfig8/current/storage/app/rconfig/scripts/keys/mysql-schema.cnf"
fail() { echo "ERROR: $1" >&2; exit 1; }
HOST="${1:-}"; DB="${2:-}"[ -n "$HOST" ] && [ -n "$DB" ] || fail "usage: db-schema.sh <db_host> <database>"[ -r "$OPTION_FILE" ] || fail "cannot read $OPTION_FILE"
echo "dumping the schema of $DB on $HOST" >&2
# --defaults-extra-file must be the first option.schema=$(mysqldump --defaults-extra-file="$OPTION_FILE" \ --host="$HOST" --no-data --routines --triggers --events \ --skip-dump-date --skip-comments --single-transaction \ "$DB") || fail "mysqldump failed for $DB on $HOST"
# AUTO_INCREMENT counters move as rows are added. They are data, not schema.schema=$(printf '%s\n' "$schema" | sed -E 's/ AUTO_INCREMENT=[0-9]+//')
[ -n "$schema" ] || fail "the schema for $DB came back empty"printf '%s\n' "$schema"echo "SIE-OK: schema of $DB on $HOST" >&2Linux server config files
Stores a list of configuration files from a Linux server, each under a delimiter line, so a change to named.conf or the RADIUS clients file shows up as a change to that server in rConfig.
It reads the files over SSH with a key. Create a dedicated account on the server that can read only those files, and put its key in the scripts keys folder.
Command:
/bin/bash {script_path}linux-files.sh {device_ip} {device_username}#!/usr/bin/env bash## Store a list of config files from a Linux server, for the rConfig Script# Integration Engine.## Usage: linux-files.sh <server> <username>
set -o pipefail
KEYS="/var/www/html/rconfig8/current/storage/app/rconfig/scripts/keys"
# Adapt for the server. Only files the account can read.FILES=( /etc/named.conf /etc/chrony.conf /etc/raddb/clients.conf /etc/ssh/sshd_config)
fail() { echo "ERROR: $1" >&2; exit 1; }
HOST="${1:-}"; USER_NAME="${2:-}"[ -n "$HOST" ] && [ -n "$USER_NAME" ] || fail "usage: linux-files.sh <server> <username>"
for file in "${FILES[@]}"; do echo "===== $file =====" ssh -i "$KEYS/files_ed25519" \ -o BatchMode=yes \ -o StrictHostKeyChecking=accept-new \ -o UserKnownHostsFile="$KEYS/known_hosts" \ -o ConnectTimeout=10 \ "$USER_NAME@$HOST" "cat -- '$file'" \ || fail "cannot read $file on $HOST" echodone
echo "SIE-OK: ${#FILES[@]} files from $HOST" >&2A file the account can’t read fails the whole run. That’s deliberate: a backup that quietly skips the file you care about is worse than one that fails.
Cloud firewall rules (AWS, Azure, Google Cloud)
Stores a cloud account’s firewall rules (AWS security groups, Azure network security groups or Google Cloud VPC firewall rules), flattened to one line per setting and sorted, so an opened port shows up as a single changed line naming the group and the rule.
One script covers all three clouds through their command-line tools: the AWS CLI (aws), the Azure CLI (az) and the Google Cloud CLI (gcloud). Install only the ones you use. Each tool normally keeps its login in the user’s home folder, which the web user usually can’t write to, so the script points each tool at a folder under the scripts keys folder instead.
This script imports the helpers from the cleaning output recipe, so save sie_clean.py in the same folder.
Signing each tool in, once
Give each cloud a read-only identity that can only list firewall rules, then sign the tool in once as the web user (www-data on Ubuntu):
| Cloud | Identity and permission | One-time sign-in |
|---|---|---|
| AWS | An IAM user or role with ec2:DescribeSecurityGroups |
Save its key in keys/aws/credentials under a profile name such as [rconfig-readonly]. If rConfig runs on EC2, an instance role needs no file at all. |
| Azure | A service principal with the Reader role on the subscription | sudo -u apache env AZURE_CONFIG_DIR=$KEYS/azure az login --service-principal -u <app-id> -p <secret> --tenant <tenant-id> |
| Google Cloud | A service account with the Compute Viewer role, or a custom role holding compute.firewalls.list |
sudo -u apache env CLOUDSDK_CONFIG=$KEYS/gcloud gcloud auth activate-service-account --key-file=<key.json> |
$KEYS is /var/www/html/rconfig8/current/storage/app/rconfig/scripts/keys. Keep those folders mode 0700, owned by the web user.
The commands
The device record’s IP address isn’t used. Make one device per account, subscription or project, with one of these commands:
/usr/bin/python3 {script_path}cloud-firewall.py aws eu-west-1 rconfig-readonly/usr/bin/python3 {script_path}cloud-firewall.py azure 00000000-0000-0000-0000-000000000000/usr/bin/python3 {script_path}cloud-firewall.py gcp example-project-idThe script
#!/usr/bin/env python3"""Store a cloud account's firewall rules, flattened and sorted, for the rConfigScript Integration Engine.
Usage: cloud-firewall.py aws <region> <profile> cloud-firewall.py azure <subscription_id> cloud-firewall.py gcp <project_id>"""
import jsonimport osimport subprocessimport sys
import sie_clean
KEYS = "/var/www/html/rconfig8/current/storage/app/rconfig/scripts/keys"TOOLS = {"aws": "/usr/local/bin/aws", "azure": "/usr/bin/az", "gcp": "/usr/bin/gcloud"} # check with: which
# Fields that change without a rule changing.DROP = {"etag", "provisioningState", "resourceGuid", "creationTimestamp", "fingerprint"}
def fail(message): print(f"ERROR: {message}", file=sys.stderr) sys.exit(1)
def run(cloud, args): env = dict(os.environ) env.update({ "AWS_SHARED_CREDENTIALS_FILE": f"{KEYS}/aws/credentials", "AWS_CONFIG_FILE": f"{KEYS}/aws/config", "AWS_PAGER": "", "AZURE_CONFIG_DIR": f"{KEYS}/azure", "CLOUDSDK_CONFIG": f"{KEYS}/gcloud", }) try: result = subprocess.run([TOOLS[cloud], *args], capture_output=True, text=True, env=env, timeout=120) except FileNotFoundError: fail(f"{TOOLS[cloud]} is not installed") if result.returncode != 0: lines = result.stderr.strip().splitlines() fail(f"{cloud} cli failed: {lines[-1] if lines else 'no error text'}") return json.loads(result.stdout or "null")
def tidy(value): if isinstance(value, dict): return {k: tidy(v) for k, v in value.items() if k not in DROP} if isinstance(value, list): return [tidy(v) for v in value] return value
def main(): if len(sys.argv) < 3 or sys.argv[1] not in TOOLS: fail("usage: cloud-firewall.py aws <region> <profile> | azure <subscription> | gcp <project>") cloud, scope = sys.argv[1], sys.argv[2]
if cloud == "aws": if len(sys.argv) < 4: fail("usage: cloud-firewall.py aws <region> <profile>") data = run(cloud, ["ec2", "describe-security-groups", "--region", scope, "--profile", sys.argv[3], "--output", "json"]) groups = {g["GroupId"]: g for g in (data or {}).get("SecurityGroups", [])} rule_lists = ("IpPermissions", "IpPermissionsEgress") elif cloud == "azure": data = run(cloud, ["network", "nsg", "list", "--subscription", scope, "--output", "json"]) groups = {g["name"]: g for g in data or []} rule_lists = ("securityRules", "defaultSecurityRules") else: data = run(cloud, ["compute", "firewall-rules", "list", "--project", scope, "--format", "json"]) groups = {g["name"]: g for g in data or []} rule_lists = ()
if not groups: fail(f"no firewall rules returned for {cloud} {scope}, refusing to store an empty backup")
# Rules in these clouds have no list order (priority is a field), so sorting is safe. for group in groups.values(): for key in rule_lists: group[key] = sorted(group.get(key, []), key=lambda r: json.dumps(r, sort_keys=True))
sys.stdout.write(sie_clean.flatten_json_text(tidy(groups))) print(f"SIE-OK: {len(groups)} {cloud} firewall groups or rules in {scope}", file=sys.stderr) return 0
if __name__ == "__main__": sys.exit(main())Groups are keyed by ID or name, so adding one doesn’t renumber the others. What stored lines look like:
sg-0a1b2c3d4e5f60001.GroupName = "web-frontend"sg-0a1b2c3d4e5f60001.IpPermissions[0].FromPort = 443sg-0a1b2c3d4e5f60001.IpPermissions[0].IpRanges[0].CidrIp = "203.0.113.0/24"allow-https-web.direction = "INGRESS"allow-https-web.sourceRanges[0] = "198.51.100.0/24"