Network SNMP Trap Handler Management - rConfig V8 CLI

SNMP Trap Handler: Real-Time Network Event Monitoring

The SNMP Trap Handler enables rConfig to receive and process SNMP trap notifications from network devices in real-time, providing immediate visibility into critical network events, configuration changes, and system alerts. This guide covers deployment, configuration, and operational management of the SNMP Trap Handler service using Supervisor for reliable, production-grade operation.

Understanding SNMP Trap Handling

How SNMP Trap Reception Works

SNMP traps are asynchronous notifications sent by network devices to inform management systems of significant events without polling. Unlike SNMP polling where the management system queries devices, traps are device-initiated, providing immediate notification of events such as:

Configuration Events: Device configuration changes, startup/reload notifications, and administrative modifications trigger traps that rConfig can correlate with configuration backups to provide audit trails and change tracking.

Operational Events: Link state changes, routing protocol updates, hardware failures, and environmental alerts generate traps that provide context for configuration analysis and troubleshooting workflows.

Security Events: Authentication failures, unauthorized access attempts, and security policy violations produce traps that complement configuration security analysis and compliance monitoring.

Integration with rConfig Workflows

The SNMP Trap Handler integrates seamlessly with rConfig’s configuration management capabilities:

Event Correlation: Traps received near configuration backup times provide context—a “configuration changed” trap followed by a backup captures the administrative action and resulting configuration in a unified timeline.

Automated Responses: Trap reception can trigger automated workflows such as immediate configuration backups, notification escalations, or compliance checks based on trap content and severity.

Historical Analysis: Trapped events stored in rConfig’s database enable historical correlation between network events and configuration changes, supporting root cause analysis and trend identification.

Prerequisites

System Requirements

Supervisor Process Manager: The SNMP Trap Handler runs as a supervised background service, requiring Supervisor for process management, automatic restart, and operational monitoring. If rConfig was installed correctly, Supervisor should already be configured. Verify installation:

# Check Supervisor status
systemctl status supervisor

# Verify Supervisor is enabled
systemctl is-enabled supervisor

Port Availability: SNMP trap reception requires a dedicated UDP port. The standard SNMP trap port (162) requires root privileges on most systems. Verify port availability:

# Check if port 162 is in use
sudo netstat -ulnp | grep :162

# Check if port 1162 (alternative) is in use
sudo netstat -ulnp | grep :1162

PHP Requirements: The trap handler requires PHP CLI with necessary extensions. Verify PHP installation:

# Check PHP version (7.4+ required)
php -v

# Verify required extensions
php -m | grep -E 'snmp|sockets|pcntl'

Network Configuration

Firewall Rules: Ensure the configured trap port accepts UDP traffic from network devices:

# For firewalld (RHEL/CentOS)
sudo firewall-cmd --add-port=162/udp --permanent
sudo firewall-cmd --reload

# For UFW (Ubuntu/Debian)
sudo ufw allow 162/udp

Device Configuration: Network devices must be configured to send traps to the rConfig server. Example Cisco configuration:

snmp-server host <rconfig-server-ip> traps version 2c public
snmp-server enable traps config
snmp-server enable traps syslog

Initial Testing and Validation

Manual Testing Procedure

Run the trap handler manually to validate configuration and connectivity:

# Navigate to rConfig installation directory
cd /var/www/html/rconfig8/current

# Test with default settings (requires root for port 162)
sudo php artisan rconfig:snmp-trap-handler --host=0.0.0.0 --port=162 --daemon

# Test with non-privileged port
php artisan rconfig:snmp-trap-handler --host=0.0.0.0 --port=1162 --daemon

# Test with specific interface binding
php artisan rconfig:snmp-trap-handler --host=192.168.1.100 --port=1162 --daemon

Testing Output: The handler displays startup information including bind address, port, and community string. Successful startup shows:

SNMP Trap Handler started
Listening on: 0.0.0.0:162
Community: public
Press Ctrl+C to stop

Sending Test Traps

Verify trap reception by sending test SNMP traps to the handler:

# Install SNMP utilities if not present
sudo apt-get install snmp  # Debian/Ubuntu
sudo yum install net-snmp-utils  # RHEL/CentOS

# Send SNMPv2c test trap
snmptrap -v2c -c public 127.0.0.1:162 "" 1.3.6.1.4.1.9.9.43.2.0.1 \
  1.3.6.1.4.1.9.9.43.1.1.1.0 s "Configuration changed by admin"

# Send SNMPv1 test trap
snmptrap -v1 -c public 127.0.0.1:162 \
  1.3.6.1.4.1.9.9.43 localhost 6 1 "" \
  1.3.6.1.4.1.9.9.43.1.1.1.0 s "Test trap message"

Verification: The trap handler displays received traps in the terminal. Example output:

[2025-10-12 14:32:15] Trap received from 127.0.0.1
  OID: 1.3.6.1.4.1.9.9.43.2.0.1
  Varbind: 1.3.6.1.4.1.9.9.43.1.1.1.0 = "Configuration changed by admin"

Port Forwarding Configuration

To run the handler without root privileges while receiving on port 162:

# Configure iptables to forward 162 → 1162
sudo iptables -t nat -A PREROUTING -p udp --dport 162 -j REDIRECT --to-port 1162

# Make forwarding persistent (RHEL/CentOS)
sudo service iptables save

# Make forwarding persistent (Ubuntu/Debian)
sudo apt-get install iptables-persistent
sudo netfilter-persistent save

Service Deployment with Supervisor

Interactive Setup (Recommended for First-Time Deployment)

The interactive management interface provides a guided workflow for service installation and configuration:

# Navigate to rConfig directory
cd /var/www/html/rconfig7/current

# Launch interactive setup
php artisan rconfig:snmp-trap-supervisor

┌─────────────────────────────────────────────────────────┐
│ SNMP Trap Handler Supervisor Management                │
├─────────────────────────────────────────────────────────┤
│ What action would you like to perform?                 │
│                                                         │
│  › 📦 Install service (add to supervisor)              │
│    🗑️  Uninstall service (remove from supervisor)      │
│    📊 Check service status                             │
│    🔄 Restart service                                  │
│    ❌ Exit                                             │
└─────────────────────────────────────────────────────────┘

Menu Options:

Install service: Configure and deploy trap handler as supervised process
Uninstall service: Remove trap handler from Supervisor (stops and deletes configuration)
Check service status: Display current operational status and statistics
Restart service: Gracefully restart the handler (applies configuration changes)
Exit: Close management interface

Basic Configuration Parameters

When installing the service, configure essential operational parameters:

Host (Bind Address):

0.0.0.0 - Listen on all network interfaces (default, most common)
192.168.1.100 - Listen on specific interface only
127.0.0.1 - Listen on localhost only (testing/debugging)

Port (UDP Listener):

162 - Standard SNMP trap port (requires root or port forwarding)
1162 - Common alternative for unprivileged operation
Custom port matching your network device configuration

Community String:

public - Default SNMP community (change for production)
Custom community matching network device trap configuration
Multiple communities supported via configuration file

Advanced Configuration Options

Select “Configure advanced settings” during installation to access:

Timeout (seconds):

SNMP socket timeout for trap reception
Default: 5 seconds
Lower values: More responsive to network issues but may drop slow traps
Higher values: More tolerant but may delay error detection

Base Path:

rConfig installation directory
Default: /var/www/html/rconfig7/current
Only modify if using non-standard installation path

Supervisor Config Directory:

Location for Supervisor configuration files
Default: /etc/supervisor/conf.d/
Alternative: /etc/supervisord.d/ (RHEL/CentOS)

Command Line Service Management

Installation and Configuration

For automated deployment or scripted installation, use command-line options:

# Install with default settings
php artisan rconfig:snmp-trap-supervisor install --force

# Install with custom basic settings
php artisan rconfig:snmp-trap-supervisor install \
  --host=192.168.1.100 \
  --port=1162 \
  --community=private \
  --force

# Install with advanced configuration
php artisan rconfig:snmp-trap-supervisor install \
  --host=0.0.0.0 \
  --port=162 \
  --community=network_monitor \
  --timeout=10 \
  --memory-limit=1024 \
  --base-path=/opt/rconfig \
  --supervisor-config-dir=/etc/supervisor/conf.d \
  --force

Parameter Reference:

--force: Overwrite existing configuration without prompting
--host: IP address or hostname to bind to
--port: UDP port for trap reception
--community: SNMP community string
--timeout: Socket timeout in seconds
--memory-limit: PHP memory limit in MB
--base-path: rConfig installation directory
--supervisor-config-dir: Supervisor configuration location

Service Control Operations

# Check current service status
php artisan rconfig:snmp-trap-supervisor status

# Restart the service (apply configuration changes)
php artisan rconfig:snmp-trap-supervisor restart

# Uninstall the service (remove from Supervisor)
php artisan rconfig:snmp-trap-supervisor uninstall

Direct Supervisor Commands

For fine-grained control, use Supervisor’s CLI directly:

# View all supervised processes
supervisorctl status

# Check SNMP trap handler specifically
supervisorctl status rconfig-snmp-trap

# Start the service
supervisorctl start rconfig-snmp-trap

# Stop the service
supervisorctl stop rconfig-snmp-trap

# Restart the service
supervisorctl restart rconfig-snmp-trap

# Reload Supervisor configuration (after manual edits)
supervisorctl reread
supervisorctl update

Monitoring and Operational Management

Real-Time Log Monitoring

Follow Logs in Real-Time (recommended for troubleshooting):

# Stream trap handler output (stdout)
supervisorctl tail -f rconfig-snmp-trap

# Stream error output (stderr)
supervisorctl tail -f rconfig-snmp-trap stderr

# View last 100 lines of output
supervisorctl tail -100 rconfig-snmp-trap

# View last 100 lines of errors
supervisorctl tail -100 rconfig-snmp-trap stderr

Log Output Examples:

[2025-10-12 14:45:23] SNMP Trap Handler initialized
[2025-10-12 14:45:23] Listening on 0.0.0.0:162
[2025-10-12 14:45:23] Community: public
[2025-10-12 14:46:15] Trap received from 192.168.1.1
[2025-10-12 14:46:15]   OID: 1.3.6.1.6.3.1.1.5.4
[2025-10-12 14:46:15]   Type: Authentication Failure
[2025-10-12 14:46:16] Trap processed and stored successfully

Service Status Verification

Check comprehensive service status:

# Via rConfig command
php artisan rconfig:snmp-trap-supervisor status

# Example output:
# ┌─────────────────────────────────────────┐
# │ Service: rconfig-snmp-trap              │
# │ Status: RUNNING                         │
# │ PID: 12345                              │
# │ Uptime: 2 days, 14:32:18                │
# │ Configuration: /etc/supervisor/conf.d/  │
# └─────────────────────────────────────────┘

# Via Supervisor directly
supervisorctl status rconfig-snmp-trap

# Example output:
# rconfig-snmp-trap    RUNNING   pid 12345, uptime 2 days, 14:32:18

Log File Locations

The trap handler automatically logs to dedicated files for persistent storage and analysis:

Standard Output Log:

tail -f /var/www/html/rconfig7/current/storage/logs/snmp-trap-handler.log

# Contains: Normal operational messages, trap reception logs, processing status

Error Output Log:

tail -f /var/www/html/rconfig7/current/storage/logs/snmp-trap-handler-error.log

# Contains: PHP errors, trap processing failures, system exceptions

Log Rotation: Configure logrotate to prevent unlimited log growth:

# Create logrotate configuration
sudo nano /etc/logrotate.d/rconfig-snmp-trap

# Add configuration:
/var/www/html/rconfig7/current/storage/logs/snmp-trap-handler*.log {
    daily
    rotate 30
    compress
    delaycompress
    notifempty
    missingok
    create 0644 www-data www-data
    postrotate
        supervisorctl restart rconfig-snmp-trap > /dev/null
    endscript
}

Testing and Validation

Comprehensive Trap Reception Testing

After service deployment, verify end-to-end trap reception and processing:

1. Service Verification:

# Confirm service is running
supervisorctl status rconfig-snmp-trap

# Check port binding
sudo netstat -ulnp | grep -E '162|1162'

2. Send Test SNMPv2c Trap:

snmptrap -v2c -c public <rconfig-server-ip>:162 "" 1.3.6.1.4.1.9.9.43.2.0.1 \
  1.3.6.1.4.1.9.9.43.1.1.1.0 s "Test: Configuration changed by admin" \
  1.3.6.1.4.1.9.9.43.1.1.2.0 s "Device: test-router-01" \
  1.3.6.1.4.1.9.9.43.1.1.3.0 i 1

3. Send Test SNMPv1 Trap:

snmptrap -v1 -c public <rconfig-server-ip>:162 \
  1.3.6.1.4.1.9.9.43 <source-device-ip> 6 1 "" \
  1.3.6.1.4.1.9.9.43.1.1.1.0 s "Test: Link state change" \
  1.3.6.1.4.1.9.9.43.1.1.2.0 s "Interface: GigabitEthernet0/1"

4. Verify Reception in Logs:

# Real-time log monitoring
supervisorctl tail -f rconfig-snmp-trap

# Or check log file
tail -20 /var/www/html/rconfig7/current/storage/logs/snmp-trap-handler.log

5. Verify Database Storage (if trap storage enabled):

# Connect to rConfig database
mysql -u rconfig -p rconfig

# Query recent traps
SELECT id, received_at, source_ip, trap_oid, description
FROM snmp_traps
ORDER BY received_at DESC
LIMIT 10;

Production Deployment Checklist

Before deploying to production, verify:

Troubleshooting

Service Won’t Start

Symptom: Service fails to start, shows “FATAL” or “BACKOFF” status in Supervisor.

Diagnostic Steps:

# Check Supervisor status
supervisorctl status rconfig-snmp-trap

# Review error logs
supervisorctl tail rconfig-snmp-trap stderr

# Check service configuration
cat /etc/supervisor/conf.d/rconfig-snmp-trap.conf

# Test manual execution
cd /var/www/html/rconfig7/current
php artisan rconfig:snmp-trap-handler --host=0.0.0.0 --port=1162 --daemon

Common Causes and Resolutions:

Invalid Base Path: Verify --base-path points to correct rConfig installation

# Correct path should contain artisan file
ls -la /var/www/html/rconfig7/current/artisan

Port Already in Use: Another process bound to trap port

# Identify conflicting process
sudo netstat -ulnp | grep :162

# Stop conflicting service or use different port

Permission Issues: Handler lacks permission to bind port or write logs

# Verify log directory permissions
ls -la /var/www/html/rconfig7/current/storage/logs/

# Fix permissions if needed
sudo chown -R www-data:www-data /var/www/html/rconfig7/current/storage/

PHP Missing Extensions: Required SNMP or socket extensions not installed

# Check for required extensions
php -m | grep -E 'snmp|sockets'

# Install if missing (Ubuntu/Debian)
sudo apt-get install php-snmp

# Install if missing (RHEL/CentOS)
sudo yum install php-snmp

Permission Denied on Port 162

Symptom: Error message “Permission denied” or “Cannot bind to port 162” when starting service.

Cause: Port 162 is privileged (< 1024), requiring root access to bind.

Resolution Options:

Option 1 - Use Port Forwarding (Recommended):

# Forward 162 → 1162 with iptables
sudo iptables -t nat -A PREROUTING -p udp --dport 162 -j REDIRECT --to-port 1162

# Configure handler for port 1162
php artisan rconfig:snmp-trap-supervisor install --port=1162 --force

# Make forwarding persistent
sudo apt-get install iptables-persistent  # Ubuntu/Debian
sudo netfilter-persistent save

Option 2 - Run as Root (Security Implications):

# Modify Supervisor config to run as root
sudo nano /etc/supervisor/conf.d/rconfig-snmp-trap.conf

# Change user directive:
user=root

# Reload Supervisor
supervisorctl reread
supervisorctl update

Option 3 - Reconfigure Devices:

# Use alternative port (no privilege issues)
php artisan rconfig:snmp-trap-supervisor install --port=1162 --force

# Update device configurations to send traps to port 1162
# Example Cisco config:
# snmp-server host 192.168.1.100 traps version 2c public udp-port 1162

Service Stops Unexpectedly

Symptom: Service shows “STOPPED” or “EXITED” status, no longer processing traps.

Diagnostic Steps:

# Check exit reason
supervisorctl status rconfig-snmp-trap

# Review error logs for crash details
supervisorctl tail rconfig-snmp-trap stderr

# Check system logs
sudo journalctl -u supervisor -n 50

# Review PHP error logs
tail -50 /var/www/html/rconfig7/current/storage/logs/laravel.log

Common Causes and Resolutions:

Memory Limit Exceeded: Handler exhausted allocated memory

# Check memory usage in error log
grep -i "memory" /var/www/html/rconfig7/current/storage/logs/snmp-trap-handler-error.log

# Increase memory limit
php artisan rconfig:snmp-trap-supervisor install --memory-limit=1024 --force
supervisorctl restart rconfig-snmp-trap

PHP Fatal Error: Code exception or unhandled error

# Review error logs for stack traces
tail -100 /var/www/html/rconfig7/current/storage/logs/snmp-trap-handler-error.log

# Update rConfig if bug identified
# Contact support with error details

Database Connection Lost: MySQL connection timeout or failure

# Verify database connectivity
mysql -u rconfig -p -e "SELECT 1"

# Check database configuration
grep DB_ /var/www/html/rconfig7/current/.env

# Restart service after database recovery
supervisorctl restart rconfig-snmp-trap

No Traps Received

Symptom: Service running but no traps appear in logs or database.

Diagnostic Checklist:

# 1. Verify service is listening
sudo netstat -ulnp | grep -E '162|1162'
# Should show: php ... listening on configured port

# 2. Test with local trap
snmptrap -v2c -c public 127.0.0.1:162 "" 1.3.6.1.6.3.1.1.5.1

# 3. Check firewall rules
sudo iptables -L -n | grep -E '162|1162'
sudo firewall-cmd --list-all  # RHEL/CentOS

# 4. Verify community string matches
# Check handler community
grep community /etc/supervisor/conf.d/rconfig-snmp-trap.conf

# 5. Test from remote device
# From network device or another host:
snmptrap -v2c -c public <rconfig-ip>:162 "" 1.3.6.1.6.3.1.1.5.1

# 6. Verify network device trap configuration
# Example verification for Cisco:
# show running-config | include snmp-server host

Resolution Path:

If local trap received: Firewall or network device configuration issue
If local trap NOT received: Handler configuration or port binding issue
If wrong community: Update handler or device configuration to match
If firewall blocking: Add rule to allow UDP traffic on trap port

High CPU or Memory Usage

Symptom: Trap handler consuming excessive system resources.

Diagnostic Steps:

# Monitor resource usage
top -p $(pgrep -f snmp-trap-handler)

# Check trap reception rate
grep "Trap received" /var/www/html/rconfig7/current/storage/logs/snmp-trap-handler.log | wc -l

# Analyze trap sources
grep "Trap received from" /var/www/html/rconfig7/current/storage/logs/snmp-trap-handler.log | \
  awk '{print $NF}' | sort | uniq -c | sort -rn | head -10

Common Causes and Resolutions:

Trap Storm: Device sending excessive traps

# Identify problematic device from log analysis
# Temporarily block device traps while investigating

# Add firewall rule to block specific source
sudo iptables -I INPUT -p udp -s <problematic-device-ip> --dport 162 -j DROP

# Investigate and resolve device issue
# Remove block after resolution

Memory Leak: Handler not releasing memory (potential bug)

# Monitor memory growth over time
watch -n 5 'ps aux | grep snmp-trap-handler'

# Implement periodic restart as workaround
# Add to crontab:
0 2 * * * /usr/bin/supervisorctl restart rconfig-snmp-trap

# Report issue to rConfig support

Complex Trap Processing: Large traps or heavy database operations

# Optimize database if trap storage enabled
mysql -u rconfig -p rconfig -e "OPTIMIZE TABLE snmp_traps;"

# Consider archiving old traps
mysql -u rconfig -p rconfig -e "DELETE FROM snmp_traps WHERE received_at < DATE_SUB(NOW(), INTERVAL 90 DAY);"

Manual Testing for Debugging: When troubleshooting, always stop the Supervisor service before running the handler manually to avoid port conflicts:

# Stop supervised service
supervisorctl stop rconfig-snmp-trap

# Run manually with verbose output for debugging
php artisan rconfig:snmp-trap-handler --host=0.0.0.0 --port=162 --daemon -vvv

# Observe detailed output for diagnostic information

# When debugging complete, stop manual process (Ctrl+C)
# Restart supervised service
supervisorctl start rconfig-snmp-trap

Performance Optimization

High-Volume Environments

For deployments receiving thousands of traps per minute:

Increase Memory Allocation:

php artisan rconfig:snmp-trap-supervisor install --memory-limit=2048 --force
supervisorctl restart rconfig-snmp-trap

Optimize Database for Trap Storage:

-- Add indexes for common queries
ALTER TABLE snmp_traps ADD INDEX idx_received_at (received_at);
ALTER TABLE snmp_traps ADD INDEX idx_source_ip (source_ip);
ALTER TABLE snmp_traps ADD INDEX idx_trap_oid (trap_oid);

-- Implement table partitioning for large datasets
ALTER TABLE snmp_traps PARTITION BY RANGE (YEAR(received_at)) (
    PARTITION p2024 VALUES LESS THAN (2025),
    PARTITION p2025 VALUES LESS THAN (2026),
    PARTITION p_future VALUES LESS THAN MAXVALUE
);

Configure Connection Pooling:

# Edit .env for database optimization
nano /var/www/html/rconfig7/current/.env

# Add or modify:
DB_CONNECTION_POOL_SIZE=20
DB_PERSISTENT_CONNECTIONS=true

Resource Monitoring

Implement continuous resource monitoring:

# Create monitoring script
sudo nano /usr/local/bin/monitor-snmp-trap.sh

#!/bin/bash
# Monitor SNMP Trap Handler resources

PID=$(pgrep -f snmp-trap-handler)
if [ -z "$PID" ]; then
    echo "CRITICAL: SNMP Trap Handler not running"
    exit 2
fi

CPU=$(ps -p $PID -o %cpu= | awk '{print int($1)}')
MEM=$(ps -p $PID -o %mem= | awk '{print int($1)}')

echo "CPU: ${CPU}% | MEM: ${MEM}%"

if [ $CPU -gt 80 ]; then
    echo "WARNING: High CPU usage"
fi

if [ $MEM -gt 50 ]; then
    echo "WARNING: High memory usage"
fi

# Make executable
sudo chmod +x /usr/local/bin/monitor-snmp-trap.sh

# Add to cron for periodic checks
*/5 * * * * /usr/local/bin/monitor-snmp-trap.sh >> /var/log/snmp-trap-monitor.log

Best Practices

Security Hardening

Use Strong Community Strings: Replace default “public” community with complex, unique strings:

php artisan rconfig:snmp-trap-supervisor install \
  --community=$(openssl rand -base64 32) \
  --force

Restrict Bind Address: Limit trap reception to specific network interfaces:

# Listen only on management network interface
php artisan rconfig:snmp-trap-supervisor install \
  --host=192.168.100.10 \
  --force

Implement Source Filtering: Configure firewall to accept traps only from known devices:

# Create chain for SNMP trap filtering
sudo iptables -N SNMP_TRAP_FILTER

# Allow traps from known network ranges
sudo iptables -A SNMP_TRAP_FILTER -s 10.0.0.0/8 -j ACCEPT
sudo iptables -A SNMP_TRAP_FILTER -s 192.168.0.0/16 -j ACCEPT

# Drop all other trap sources
sudo iptables -A SNMP_TRAP_FILTER -j DROP

# Apply filter to trap port
sudo iptables -A INPUT -p udp --dport 162 -j SNMP_TRAP_FILTER

Operational Excellence

Implement Monitoring and Alerts: Use monitoring tools (e.g., Nagios, Zabbix) to track service health and receive alerts on failures. Regularly Review Logs: Schedule periodic log reviews to identify anomalies or unusual trap patterns. Backup Configuration: Regularly back up rConfig and Supervisor configurations to prevent data loss. Test Disaster Recovery: Periodically test recovery procedures to ensure rapid restoration of service in case of failure.

Conclusion

The SNMP Trap Handler, managed via Supervisor, provides rConfig with robust real-time network event monitoring capabilities. By following this comprehensive guide, administrators can deploy, configure, and maintain the trap handler service effectively, ensuring reliable reception and processing of SNMP traps to enhance network visibility and operational workflows. Regular testing, monitoring, and adherence to best practices will ensure optimal performance and security in production environments.