Advanced Troubleshooting Techniques
Advanced Troubleshooting Techniques
Section titled “Advanced Troubleshooting Techniques”When the basic checks pass but backups still fail, the root cause is usually one of: a timeout misconfiguration, a stalled interactive prompt the script cannot answer, or a protocol or vendor-specific quirk.
When to use this
Section titled “When to use this”Reach for this guide after you have confirmed connectivity, authentication, and prompt configuration, but backups still fail intermittently or for specific device types. It covers timeout analysis, stalled connections, protocol quirks, and vendor-specific behaviours that are not addressed in the earlier troubleshooting pages.
Prerequisites
Section titled “Prerequisites”- Basic connectivity confirmed (see Basic Connectivity Verification)
- Authentication working against the device (see Authentication Verification)
- Prompt configuration verified (see Device Prompt Troubleshooting)
- Debug output captured using the rConfig CLI Diagnostic Tools or Live Debug Console
Analysing Timeout Issues
Section titled “Analysing Timeout Issues”Timeouts occur when rConfig does not receive the expected prompt within the timeout period (default: 60 seconds).
-
Run the debug command and observe where the timeout occurs:
- During connection: network or service availability issue
- During authentication: slow external authentication provider (TACACS, RADIUS, LDAP)
- During prompt detection: prompt mismatch or device not displaying the expected prompt
- During command execution: command output exceeds the timeout period
-
Identify the cause from timing in the debug output:
- Connection takes 30+ seconds: network latency or routing problem
- Authentication takes 20+ seconds: slow external authentication provider
- Prompt detection never completes: prompt configuration is incorrect
- Command execution exceeds timeout: large configuration or a slow device processor
-
Apply the matching resolution:
Increase the timeout for devices with large configurations or slow processors. Edit the device’s connection template and raise
connect.timeout. The recommended range is 5-30 seconds: 5-10 seconds for fast devices, 20-30 seconds for slower ones or those with large configs.Optimise commands if the timeout is command-specific. Replace
show tech-supportwithshow running-config, or remove diagnostic commands from the command set.Fix prompt configuration if the timeout occurs while waiting for a prompt that never appears. See Device Prompt Troubleshooting.
Reduce authentication latency if TACACS+ or RADIUS is slow: use a local device account for rConfig, configure fallback to local authentication, or optimise the authentication server.
Handling Stalled Connections
Section titled “Handling Stalled Connections”A stalled connection stops producing output without triggering a timeout error.
Symptom: Debug output shows the connection progressing normally, then stops with no further output, no timeout, and no error.
Interactive configuration prompt: Some devices display a setup wizard on first login:
The system has detected that you are logging in for the first time.Would you like to enter basic configuration? [yes/no]:rConfig cannot answer interactive prompts. Disable initial configuration prompts on the device for automated sessions.
Banner requiring acknowledgment:
*** WARNING: Unauthorized access prohibited ***Press any key to continue...The device expects a keypress before displaying the prompt, but rConfig cannot detect or respond to this. Remove the interactive banner or replace it with a non-interactive version.
Privilege escalation prompt with no credentials:
Password:The device requires an enable password, but none is configured in rConfig. Configure the enable password in the device credentials, or use an account that does not require privilege escalation.
Protocol-Specific Issues
Section titled “Protocol-Specific Issues”Host key verification failures: rConfig automatically accepts unknown host keys, but a corrupted known_hosts entry can cause failures. Remove the stale entry:
ssh-keygen -R 192.168.1.1Cipher negotiation failures: Some devices support only legacy SSH ciphers not offered by modern SSH clients. Configure the SSH client to allow legacy ciphers, or upgrade the device firmware.
Key exchange algorithm incompatibility: Older devices may not support the key exchange algorithms in modern SSH clients. Configure legacy key exchange algorithms or upgrade the device firmware.
Telnet
Section titled “Telnet”Protocol negotiation failures: Telnet negotiation can fail with some device implementations, causing hangs or garbled output. Migrate to SSH for better reliability and security wherever possible.
Character encoding issues: Special characters or control codes in device output can be misinterpreted. Review command output and modify the command set to avoid problematic characters.
Vendor-Specific Behaviours
Section titled “Vendor-Specific Behaviours”Cisco IOS: Generally reliable SSH and Telnet. Watch for privilege level issues requiring an enable password. VTY line configuration limits concurrent sessions.
Juniper Junos: Default CLI is shell-style; the device may need configuration to stay in operational mode. Prompts change based on hierarchy level. Use | display set format for consistent output.
Palo Alto Networks: API access is preferred over SSH for configuration retrieval. SSH sessions may time out on large configurations.
Fortinet FortiGate: Configuration output format varies by FortiOS version. Prompts include spaces and special formatting that require careful regex. Some commands need confirmation or interactive input.
Arista EOS: Generally excellent SSH compatibility. Prompt customisation is common, so use flexible regex patterns.
Consult vendor-specific documentation and rConfig’s vendor configuration guides for detailed troubleshooting of vendor-specific issues.
What’s next
Section titled “What’s next”- Workflow, Best Practices and Escalation - End-to-end troubleshooting workflow and what to provide when escalating to support
- rConfig CLI Diagnostic Tools - Capture verbose protocol output to identify where failures occur
- Live Debug Console - Stream debug output in the browser for a faster feedback loop