Data Purge, Config Retention & History Repair in rConfig 8.1

Data Purge, Config Retention & History Repair (rConfig 8.1)

rConfig 8.1 introduces a modern, highly controlled approach to database cleanup, configuration file retention, and self-healing repair of configuration history. This system replaces multiple older purge commands from rConfig 7.x and earlier.

Important:

• The old commands rconfig:purge-configs and rconfig:purge-configs-v2 are deprecated.
• Any scheduled cron tasks referencing these commands must be removed.
• rConfig 8.1 automatically disables these legacy tasks at runtime.

All retention, cleanup, and repair functionality now lives in three coordinated subsystems:

1. Data Purge Service (generic tables)
2. Config Retention Service (configs + config_changes + file cleanup)
3. Config History Repair Service (repair broken version sequences)

You control all of this from:

Settings → Data Purge (/settings/data-purge)

This UI lets you:

• enable/disable purge per table
• change retention days
• configure options
• inspect Purge Run History
• manually trigger repair tools (where applicable)

---

Overview of the Three Components

1. DataPurgeService

Handles all non-config tables such as:

• activity_log
• jobs
• failed_jobs
• agent_logs (Vector)
• agent_queues (Vector)
• other high-volume tables

This service respects:

• retention days
• batch size
• queue thresholds
• frequency (daily/weekly/monthly)
• dry-run mode

Workflow (non-config tables)

Notes

• Non-config tables may be deleted directly or via queued jobs.
• rConfig never queues purge jobs when running in --dry-run.

---

2. ConfigRetentionService

A specialised retention engine dedicated to:

• configs
• config_changes
• non-downloaded configs
• config file deletion
• protected version logic

Key Safety Features

• Always keeps the final latest_version=1 config
• Ignores disabled devices
• Keeps the last N versions (min_versions_per_device_command)
• Deletes corresponding config_changes safely
• Validates timestamp cutoffs
• Never queues config purge jobs (always direct execution)

Workflow

---

3. ConfigHistoryRepairService

This engine repairs broken version chains, which can happen after years of retention, manual operations pre-8.x, or inconsistent incrementing.

What It Fixes

• Latest version flags
• Gaps or resets in version sequences
• Missing previous_config_id
• config_change.version mismatches
• Chains that restart after purges (e.g., version jumps from 254 → 1,2,3…)

Workflow

---

Using the Settings UI (/settings/data-purge)

This is now the primary entry point for all purge and repair configuration.

What You Can Configure

Setting	Description
Retention Days	Number of days to keep records
Enabled	Enable/disable purge for this table
Batch Size	Size of deletion batches
Cleanup Frequency	Daily / Weekly / Monthly
min_versions_per_device_command	For configs only — number of versions to always keep
non_downloaded_retention_days	Optional separate retention for config_downloaded=0

Purge Runs Table (History)

Also located on /settings/data-purge, this shows:

• when purges ran
• dry-run vs real
• counts of deleted records
• error messages
• job queue involvement
• context (data_purge, config_retention, repair tools)

This is crucial for debugging customer issues.

---

CLI Usage

1. Full Purge (all tables)

Dry-run:

php artisan rconfig:data-purge --dry-run

Real:

php artisan rconfig:data-purge

---

2. Purge Specific Table

Dry-run:

php artisan rconfig:data-purge --table=activity_log --dry-run

Real:

php artisan rconfig:data-purge --table=activity_log

---

3. Config Retention Only

Dry-run:

php artisan rconfig:data-purge --table=configs --dry-run

Real:

php artisan rconfig:data-purge --table=configs

Note: Config retention never queues jobs. It’s always processed immediately and safely.

---

4. Repair Config History

Dry-run:

php artisan rconfig:repair-config-history --device-id=123 --dry-run

Real:

php artisan rconfig:repair-config-history --device-id=123

Repair all devices:

php artisan rconfig:repair-config-history --all

---

Understanding Job Queues

When Jobs ARE Queued

Only when purging non-config tables, and when:

• It’s NOT dry-run
• record_count >= queue_threshold
• purge is NOT forced direct
• not using --no-queue
• queue driver is not sync

Jobs are dispatched to:

PurgeDataJobs

View in Horizon:

/horizon

---

Migration Notes (V7 → V8.1)

Remove Old Purge Commands

If your system has cron entries for:

• rconfig:purge-configs
• rconfig:purge-configs-v2

Remove them.

rConfig 8.1 disables these automatically, but legacy cron jobs may still call them.

Why This Change Was Needed

Old purge commands:

• caused version chain resets
• left orphaned config_changes
• didn’t delete files safely
• didn’t respect protected versions
• weren’t integrated with the UI
• had no reporting or auditing

The new system fixes all of this.

---

Examples

Example 1 — Keep 5 versions per device+command

The setting should be:

{
  "enabled": true,
  "min_versions_per_device_command": 5,
  "batch_size": 1000,
  "cleanup_frequency": "daily"
}

---

Example 2 — Purge all logs older than 90 days

UI:

• activity_log → retention_days = 90

CLI:

php artisan rconfig:data-purge --table=activity_log

---

Example 3 — Purge only non-downloaded configs after 7 days

{
  "non_downloaded_retention_days": 7,
  "min_versions_per_device_command": 3
}

---

Troubleshooting

“Nothing is deleting”

Checklist:

• Is purge disabled?
• Retention days = 0?
• Not scheduled to run today?
• No timestamp column?
• Dry-run mode still on?
• Config purge always direct, never queued

---

“Jobs aren’t appearing in Horizon”

• configs never queue
• dry-run never queues
• threshold not reached
• force-queue not used
• queue driver = sync
• Horizon not running

---

“Version numbers reset to 1,2,3 after purge”

Run:

php artisan rconfig:repair-config-history --all

Then:

php artisan rconfig:data-purge --table=configs --dry-run

---

Related Documentation

• CLI Commands
• Device Management
• Universal Device Support
• Encryption for Config Files
• Troubleshooting Backups