Skip to content

rConfig Vector – Adding Agents

8 mins Vector

Introduction

Before installing Vector, you need to add agents to your Vector configuration. Agents are the active software components that collect configurations and metrics from your network systems and send them to the Vector server. Setting up an agent on the Vector Server is simple, but there are a few decisions to make on a per agent basis.

Agent overview

How an agent connects to the server

Agents connect to the Vector server using a secure connection. Several security mechanisms MUST be in place to ensure that the agent is connecting to the correct server and that the data is encrypted in transit. The Vector Server must be routable from the agent, and the agent must be able to connect to the server on the configured port. A the Vector Server must be expose a HTTPS port to the agent, either on private or public networks. A valid certificate must be installed on the Vector Server to ensure that the agent can connect securely. Self signed certificates are not supported, and out of date TLS or certificates will cause agents to fail to connect. All connections from the agent to the server are initiated by the agent, and the server does not initiate connections to the agent.

How and Agent is secured and authenticated

Aside from SSL connection requirement, when setting up an Agent on the Vector Server two essential security parameters are required. The Source IP address of the agent, and an API Token. DEpending on if the Agent is Natted outbound, the source IP address can be a single IP, or a range of IP addresses. it could also be the source IP of ths host itself. The API Token is a unique token that is generated by the Vector Server, and is used to authenticate the agent to the server. The API Token is a shared secret between the agent and the server, and must be kept secure. The API Token is used to authenticate the agent to the server. Data in transit is encrypted using the SSL connection, and the API Token is used to authenticate the agent to the server.

How an agent is configured

Agents are configured using a configuration file that is generated by the Vector Server. The configuration file contains the server IP address, the port to connect to, the API Token, and the source IP address of the agent. The configuration file is generated by the Vector Server, and is downloaded by the agent. The agent reads the configuration file, and uses the information in the file to connect to the server. The configuration file is unique to each agent, and is generated by the Vector Server when the agent is added to the server. You can download this file and copy its contents to the agent. Check the agent installation guide for more information on how to install the agent with its configuration file.

How an agent operates

The Vector Agent is a service that efficiently manages log synchronization, job processing, and settings synchronization. Designed for scalability and reliability, it ensures smooth API interactions and supports graceful shutdowns for continuous operation.

Key Features

  • Startup & Initialization: Sets up directories, logging, and connections to databases.
  • API Integration: Checks API status and syncs settings periodically.
  • Log Synchronization: Syncs logs to a persistent database.
  • Job Queue Management: Dynamically manages workers to process jobs efficiently.
  • Graceful Shutdown: Handles termination signals and cleans up resources.

Main Execution Flow

  1. Startup: Initializes components, directories, tickers, and connects to databases.
  2. API Interaction: Checks API status and starts settings synchronization.
  3. Log Syncing: Continuously syncs logs with the database.
  4. Job Processing: Dynamically scales worker pools to process queued jobs.
  5. Shutdown: Waits for ongoing tasks to complete before shutting down.

Key Components

  • Initialization: Sets up directories, settings, and databases.
  • Signal Handling: Monitors system signals for graceful termination.
  • Worker Pool Management: Dynamically adjusts worker pools to match workload.
  • Logging: Structured logging for efficient monitoring and troubleshooting.

Configurable Timing and Sync Intervals

  • The Vector Agent relies on configurable time intervals for critical processes, such as:
    • Log Synchronization Interval: Determines how frequently logs are synced to the database.
    • Job Queue Polling Interval: Specifies how often the agent checks for new jobs.
    • Settings Sync Interval: Manages how frequently the agent syncs configuration settings from the server.
  • These intervals and timers are configurable on the Vector server side and are retrieved during the initial API integration sync. This allows dynamic updates without restarting the agent, ensuring that timing parameters adapt to changing workloads and server-side configurations.

Observability and Reliability

  • Structured Logs: Provides traceability and insights into key events.
  • Error Handling: Manages database failures, API connectivity, and job retries.
  • Dynamic Scaling: Adjusts job workers based on system load and API feedback.

Future Improvements

  • Enhanced metrics for performance monitoring.
  • Improved retry mechanisms for failed jobs.
  • Dynamic configuration updates at runtime.
  • Additional protocols for backups and telemetry.

Agent Parameters

The Vector Agent retrieves the following configurable parameters from the Vector server during the API integration sync. These parameters are dynamically consumed and adjusted by the agent without requiring restarts:

  1. Agent Sync Interval: Time (in seconds) between agent check-ins with the server.
  2. Queue Download Rate: Rate (in seconds) for polling the job queue from the Vector Server to download new tasks to the agent.
  3. Log Upload Rate: Frequency (in seconds) at which local agent logs are uploaded to the Vector Server.
  4. Worker Count: Number of workers processing jobs in the agent. This value determines the parallelism of job processing. After adjust this value, the agent will automatically adjust the number of workers to match the workload on the next Agent Sync Interval.
  5. API Retry Interval: Time (in seconds) between retries API calls to the Vector Server for failed operations.
  6. API Retry Count: Number of retry attempts for failed operations.
  7. Max Missed Checkins: Maximum number of missed check-ins before triggering an alert and invoke an Agent failure.
  8. Agent Debug: Debug mode toggle for logging additional details when troubleshooting.

4. Adding an agent

Log in to the Vector Server and navigate to the Settings > Agents page. Click on the “Add Agent” button to add a new agent. You will be prompted to enter the following details:

  1. Name: (required)
  2. Email: (optional)
  3. Source IP: (required)

All other options are set with defaults, and should be adjusted as needed. The full list, their intentions and value types are in the previous section. Click on the “Save” button to create the agent. The agent will be added to the server, and a configuration file will be generated. Download the configuration file and copy its contents to the agent. The agent will now be able to connect to the server and send data.

Screenshot showing Vector agent add form

Conclusion

Adding agents to the Vector Server is a simple process, but it is essential to ensure that the agent is correctly configured and secured. Agents connect to the server using a secure connection, and the agent must be able to connect to the server on the configured port. The agent is authenticated using an API Token, and the source IP address of the agent must be configured on the server. Once the agent is added to the server, a configuration file is generated, and the agent can connect to the server and send data.

Check out the Troubleshooting section for more information on how to diagnose and fix common issues with agents.