System Logs

System Logs: Framework-Level Debugging and Troubleshooting in rConfig V8

The System Log Viewer provides an elegant interface for examining application-level logs without requiring command-line access. This addresses the challenge of troubleshooting complex issues where traditional database-driven application logs lack the technical depth needed for diagnosing framework-level errors, queue processing failures, or system exceptions.

Organizations can leverage the System Log Viewer to investigate Laravel framework errors, review detailed stack traces, monitor queue worker output, and diagnose issues that occur below the application layer. Unlike the Application Log which captures business logic events, System Logs contain the raw technical output from the rConfig framework, web server, and background processes.

Understanding System Logs vs Application Logs

When to Use System Logs

System Logs serve a different purpose than the Application Log:

System Logs contain:

• Laravel framework errors and exceptions with full stack traces
• PHP fatal errors and warnings
• Queue worker processing details and failures
• HTTP request/response debugging information
• Database query errors with full SQL statements
• Framework-level performance profiling
• Background job execution details

Application Log contains:

• Business logic events (device connections, task executions)
• User authentication and authorization events
• Configuration download outcomes
• Scheduled task results
• Security audit trail

Use System Logs when troubleshooting technical failures like “white screens,” unexpected exceptions, or when rConfig support requests detailed error information. Use Application Logs when investigating operational issues like device connection failures or task scheduling problems.

How System Logs Work

The System Log Viewer interfaces with Laravel’s logging system, which writes structured log files to the filesystem:

1. Log generation: The Laravel framework and rConfig application write log entries to files in /var/www/html/rconfig8/current/storage/logs/
2. File-based storage: Logs are stored as dated files (e.g., laravel-2025-10-15.log) with automatic daily rotation
3. Structured format: Each log entry includes timestamp, severity level, message, context, and stack trace when applicable
4. Web-based viewing: The Log Viewer application parses these files and presents them in a searchable, collapsible interface
5. Log rotation: Laravel automatically creates new log files daily, with older files accumulating until manually deleted

This file-based approach complements the database-driven Application Log by providing deeper technical detail that would be impractical to store in database tables.

Log Viewer Features

The System Log Viewer transforms plain text log files into an interactive interface:

Formatted display: Log entries are presented with syntax highlighting, making stack traces and error messages easier to read and comprehend

Collapsible entries: Large stack traces and debug information can be collapsed to reduce visual clutter, then expanded when detailed investigation is needed

Search functionality: Keyword search across log contents enables rapid location of specific errors, exceptions, or events

Bulk operations: Delete multiple log files simultaneously to reclaim disk space

Download capability: Export log files for offline analysis or attachment to support requests

Severity filtering: Focus on specific log levels (emergency, alert, critical, error, warning, info, debug)

Prerequisites

Access to the System Log Viewer requires:

• Admin role: Only users with the ‘admin’ role can access System Logs due to their technical sensitivity
• Web server access: The rConfig web interface must be accessible
• Filesystem permissions: The web server must have read access to /var/www/html/rconfig8/current/storage/logs/

Viewing System Logs

Accessing the Log Viewer

To access the System Log Viewer:

1. Navigate to System Logs from the main navigation menu
2. The interface displays a list of all available log files, organized by date (newest first)
3. Each log file entry shows:
   • Filename: Typically laravel-YYYY-MM-DD.log
   • File size: Indicates log volume for that day
   • Date modified: Last write time to the log file
   • Actions menu: Three-dot menu for file operations

Viewing Log Contents

To examine a specific log file:

1. Click on the log filename from the main list
2. The log contents display in formatted view with entries organized by timestamp (newest first)
3. Each log entry shows:
   • Timestamp: Precise date and time with timezone
   • Severity level: Color-coded indicator (red for errors, yellow for warnings, etc.)
   • Log message: Primary error or event description
   • Context: Additional data relevant to the event
   • Stack trace: For exceptions, the full call stack showing code execution path
4. Click on any entry to expand or collapse detailed information

Searching Log Contents

To locate specific errors or events:

1. Open the desired log file
2. Use the search field at the top of the log viewer
3. Enter search terms related to the issue:
   • Error messages: Connection refused, timeout, permission denied
   • Class names: DeviceConnectionService, CommandExecutor
   • Exception types: PDOException, ConnectionException, TimeoutException
   • Device identifiers: Hostnames, IP addresses, or device IDs
4. The viewer highlights matching entries and scrolls to the first match
5. Use browser find (Ctrl+F or Cmd+F) for additional search capabilities

Search Tips:

Search for partial exception class names to find all related errors. For example, searching ConnectionException finds SSHConnectionException, TelnetConnectionException, and SNMPConnectionException.

When investigating device-specific issues, search for the device hostname or IP address to see all log entries related to that device’s operations.

Downloading Log Files

To export log files for offline analysis or support requests:

1. Locate the log file in the main list
2. Click the three-dot menu icon next to the filename
3. Select Download from the menu
4. The log file downloads to your local system in plain text format
5. Attach downloaded logs to support tickets or share with team members for collaborative troubleshooting

Downloaded log files retain their original format and can be opened in any text editor for analysis. When sending logs to rConfig support, always download and attach the complete log file rather than copying excerpts, as context surrounding errors is often critical for diagnosis.

Managing Log Files

To delete log files and reclaim disk space:

Delete individual files:

1. Click the three-dot menu next to the log filename
2. Select Delete from the menu
3. Confirm the deletion when prompted

Delete multiple files:

1. Select the checkboxes next to multiple log filenames
2. Click the bulk action menu at the top of the list
3. Select Delete selected from the menu
4. Confirm the bulk deletion when prompted

Caution

Deleted log files cannot be recovered. Before deleting logs, ensure they are not needed for ongoing investigations or compliance requirements. Consider downloading important logs for archival before deletion.

Common Log Analysis Scenarios

Investigating “White Screen” or 500 Errors

When the rConfig interface displays a blank white screen or HTTP 500 error:

1. Access the System Log Viewer
2. Open today’s log file (most recent)
3. Scroll to entries near the time the error occurred
4. Look for log entries with error or critical severity
5. Examine the exception type and message for clues:
   • PDOException suggests database connectivity issues
   • PermissionDeniedException indicates file system permission problems
   • FatalErrorException often relates to PHP memory or timeout limits
6. Review the stack trace to identify which code path triggered the error
7. Note the complete error message and stack trace for support requests

Diagnosing Queue Job Failures

When scheduled tasks or background operations fail silently:

1. Access the System Log Viewer
2. Open the log file corresponding to when the job was scheduled
3. Search for the job class name (e.g., DeviceConnectionJob, ConfigBackupJob)
4. Look for entries with error severity containing the job name
5. Review the exception message to identify the failure cause:
   • Database deadlocks during high concurrency
   • Timeout exceptions for long-running operations
   • Connection failures to external systems
6. Check if the job was retried by searching for multiple entries with the same job ID
7. Verify queue worker health by looking for continuous processing activity

Tracking Database Query Issues

When experiencing slow performance or database errors:

1. Access the System Log Viewer
2. Search for QueryException or PDOException
3. Examine the SQL queries in the exception context
4. Look for patterns indicating problems:
   • Long-running queries without proper indexes
   • Deadlock errors suggesting transaction conflicts
   • Connection pool exhaustion during high load
5. Note the query parameters and table names for optimization efforts
6. Review the frequency of similar query failures to identify systemic issues

Troubleshooting

Symptom: Log Viewer Shows No Log Files

Diagnosis: Verify log directory exists and contains files.

Possible Causes:

• Log directory does not exist or was accidentally deleted
• Web server lacks read permissions on log directory
• Laravel logging disabled in configuration
• No errors have occurred yet to generate logs

Resolution Steps:

1. Verify log directory exists:

   ls -la /var/www/html/rconfig8/current/storage/logs/

2. Check directory permissions:

   sudo chown -R www-data:www-data /var/www/html/rconfig8/current/storage/logs/
   sudo chmod -R 755 /var/www/html/rconfig8/current/storage/logs/

3. Verify Laravel logging configuration:

   grep LOG_CHANNEL /var/www/html/rconfig8/current/.env

4. Trigger a test log entry:

   php /var/www/html/rconfig8/current/artisan tinker
   Log::info('Test log entry');

5. Refresh the Log Viewer interface to see if the test entry appears

Symptom: Cannot Access Log Viewer (Permission Denied)

Diagnosis: Verify user role and permissions.

Possible Causes:

• User account does not have ‘admin’ role
• Role-based access control misconfigured
• Session expired, requiring re-authentication

Resolution Steps:

1. Log out and log back in to refresh session permissions

2. Verify your user account has the ‘admin’ role by checking the user management interface

3. If you should have admin access but don’t, contact an existing administrator to update your role

4. Review audit logs to confirm your user account permissions

Symptom: Log Files Growing Excessively Large

Diagnosis: Check log file sizes and logging configuration.

Possible Causes:

• Debug-level logging enabled in production
• Repeated errors generating thousands of log entries
• Missing log rotation or cleanup automation
• Application stuck in error loop generating continuous logs

Resolution Steps:

1. Check current log file sizes:

   ls -lh /var/www/html/rconfig8/current/storage/logs/

2. Verify log level configuration is appropriate for production:

   grep LOG_LEVEL /var/www/html/rconfig8/current/.env

3. Set log level to ‘info’ or ‘warning’ for production environments:

   LOG_LEVEL=info

4. Review the most recent log entries to identify repeated errors:

   tail -100 /var/www/html/rconfig8/current/storage/logs/laravel.log

5. Address the underlying issue causing repeated errors

6. Delete old log files through the Log Viewer interface or via command line:

   find /var/www/html/rconfig8/current/storage/logs/ -name "laravel-*.log" -mtime +30 -delete

Symptom: Log Viewer Interface Loads Slowly

Diagnosis: Check log file size and server resources.

Possible Causes:

• Individual log files exceed several hundred megabytes
• Server has insufficient memory to load large files
• Web server PHP memory limit too low
• Multiple large log files loading simultaneously

Resolution Steps:

1. Check size of the current day’s log file:

   ls -lh /var/www/html/rconfig8/current/storage/logs/laravel-$(date +%Y-%m-%d).log

2. If log file exceeds 100MB, download it for offline viewing instead of viewing in browser

3. Increase PHP memory limit if needed:

   sudo nano /etc/php/8.2/fpm/php.ini

   Set memory_limit = 512M or higher

4. Restart PHP-FPM:

   sudo systemctl restart php8.2-fpm

5. Delete old log files to reduce the total file list

6. Consider implementing automated log cleanup for future prevention

Best Practices

Security

Restrict access to admin users only: System logs contain sensitive technical information including database connection strings, API keys in error contexts, and internal system architecture details. Ensure only administrators with legitimate troubleshooting needs can access System Logs.

Sanitize logs before sharing externally: When sending log files to vendors or external support, review them for sensitive data. Consider redacting IP addresses, hostnames, credentials, or proprietary information before distribution.

Implement log retention policies: While System Logs are valuable for troubleshooting, indefinite retention creates security risks. Establish policies to automatically delete logs older than 90-180 days unless required for compliance.

Monitor for security events in logs: Regularly review System Logs for security indicators such as authentication bypass attempts, SQL injection patterns in exception messages, or suspicious file access errors.

Performance

Set appropriate log levels for environments: Use ‘debug’ level logging only in development or when actively troubleshooting specific issues. Production environments should use ‘info’ or ‘warning’ levels to reduce log volume and disk I/O overhead.

Implement automated log cleanup: Create scheduled tasks to delete log files older than your retention policy automatically. Without cleanup, log directories can consume hundreds of gigabytes over months.

Monitor disk space on log partition: Configure alerts when the partition containing /var/www/html/rconfig8/current/storage/logs/ reaches 80% capacity. Full disk conditions prevent new logs from being written and can crash the application.

Download large logs rather than viewing in browser: When log files exceed 50MB, download them and use command-line tools (grep, less, awk) for analysis. Browser-based viewing of large files consumes significant memory and can timeout.

Organization

Establish log review procedures: Define schedules for reviewing System Logs as part of routine maintenance. Weekly review of error-level entries identifies recurring issues before they impact operations.

Correlate System Logs with Application Logs: When investigating complex issues, cross-reference System Log timestamps with Application Log events. The combination provides both technical detail and business context.

Document common error patterns: Maintain internal documentation mapping frequent exception types to known causes and resolutions. This accelerates troubleshooting and reduces dependency on external support for routine issues.

Archive critical logs before deletion: Before deleting log files during cleanup, identify any containing critical incidents or security events. Archive these logs to separate storage for future reference or compliance purposes.

Maintenance

Review log levels quarterly: As deployments mature and stabilize, logging requirements change. Quarterly review ensures log levels remain appropriate and don’t generate excessive unnecessary detail.

Test log rotation and cleanup procedures: Verify that log rotation occurs correctly and old logs are properly deleted according to retention policies. Failed rotation can lead to single log files growing to multiple gigabytes.

Verify log directory permissions after system updates: OS updates or PHP version changes can reset directory permissions. After maintenance, confirm the web server can still write to the log directory.

Include logs in backup strategies selectively: While Application Log data in the database should be backed up, System Logs on the filesystem may not require backup unless they contain information relevant to compliance or long-term reference.

Related Documentation

• Application Log - Database-driven operational and security event logs
• Horizon Queue Manager - Background job processing and queue worker monitoring
• Troubleshooting Guide - Common issues and diagnostic procedures
• User Management - Role-based access control configuration

Quick Reference

Log Severity Levels

Severity	When to Use	Typical Contents
emergency	System unusable	Critical application crashes
alert	Immediate action required	Database unavailable, service failure
critical	Critical conditions	Application component failures
error	Error conditions	Exceptions, failed operations
warning	Warning conditions	Deprecated feature usage, approaching limits
info	Informational	Routine operations, successful tasks
debug	Debug-level messages	Detailed execution flow, variable dumps

Common Exception Types

Exception	Typical Cause	Where to Investigate
PDOException	Database connectivity or query issues	Database configuration, network connectivity
ConnectionException	Device or service connection failures	Network firewall rules, device availability
TimeoutException	Operation exceeded time limit	System resources, network latency
PermissionDeniedException	File system or application permissions	Directory ownership, file permissions
FatalErrorException	PHP fatal error (memory, syntax)	PHP configuration, code issues
ValidationException	Invalid input data	Input validation logic, data format

Useful Command-Line Log Analysis

tail -f /var/www/html/rconfig8/current/storage/logs/laravel.log

grep -i "exception" /var/www/html/rconfig8/current/storage/logs/laravel-$(date +%Y-%m-%d).log

grep "ERROR" /var/www/html/rconfig8/current/storage/logs/laravel.log | wc -l

find /var/www/html/rconfig8/current/storage/logs/ -name "laravel-*.log" -mtime +30 -ls

du -sh /var/www/html/rconfig8/current/storage/logs/

Summary

The System Log Viewer provides essential technical visibility into Laravel framework operations, PHP execution, and background job processing. Organizations can leverage this tool to diagnose complex technical failures, investigate performance issues, and provide detailed error information to support teams when assistance is needed.

Key takeaways for effective system log management:

• Use System Logs for technical issues like exceptions, framework errors, and queue failures, while using Application Logs for operational events
• Download large logs for offline analysis rather than viewing in the browser to avoid performance issues
• Set appropriate log levels based on environment (debug for development, info or warning for production)
• Implement automated cleanup to prevent disk space exhaustion from log accumulation
• Restrict access to admins due to sensitive technical information contained in system logs

System Logs complement the Application Log by providing deeper technical detail necessary for diagnosing framework-level issues. Together, these logging systems enable comprehensive visibility across both application and infrastructure layers, empowering operations teams to maintain reliable network configuration management operations.

Note

Credit to Arnas Skirius for the Log Viewer application integrated into rConfig V8.