What is log aggregation?
Log aggregation is the practice of forwarding log records from many sources to a central collector, usually moments after they are written, so they can be stored, searched and analyzed together. Without it, an investigator has to sign in to hundreds of machines one at a time, each keeping its own small log that overwrites itself. With it, a single query can show a user's VPN sign-in, the file server they reached and the firewall connection that followed. Aggregation is the foundation a SIEM is built on, but it also feeds log management platforms, data lakes and simple syslog servers.
How logs reach the collector
- Syslog: Linux and Unix hosts, firewalls, switches and most appliances send messages to a collector, traditionally over UDP 514, with TLS on TCP 6514 for confidentiality and reliable delivery.
- Agents and forwarders: software on each host reads local logs, including Windows event logs and application files, buffers them and ships them on.
- Windows Event Forwarding: built-in subscriptions send selected Windows events to a Windows Event Collector server over WinRM, with no third-party software.
- APIs: connectors pull audit logs from cloud platforms and SaaS applications that offer no log file to read.
On a Linux host running rsyslog, one line forwards everything:
*.* @@logs.corp.example:514
The double at sign means TCP; a single one means UDP. Plain TCP improves delivery but still sends the messages in cleartext, so sensitive environments configure rsyslog's TLS transport instead.
What happens after collection
Raw logs arrive in dozens of formats, so the pipeline parses each one into common fields such as time, host, user, source address and action, a step called normalization. It adds context, such as the asset's owner or an address's location, and drops or summarizes noisy events to control cost. Timestamps from different sources only line up if every device synchronizes its clock with NTP, and recording times in UTC avoids daylight saving errors. The data is then indexed for fast search and later moved to cheaper archive storage for the rest of its retention period.
Why it matters for security
The strongest reason is that evidence survives. Attackers with administrative access often clear local logs; clearing the Windows Security log generates event 1102. If the entries were already forwarded, the central copy still shows what happened before the wipe, and the 1102 event becomes an alert in its own right. Aggregation also makes cross-source correlation possible, lets one retention policy cover everything, and exposes gaps: a source that stops sending is worth an alert, because it may mean a broken agent or one an attacker stopped.
Log aggregation vs. SIEM
Log aggregation collects and stores. A SIEM adds analysis on top: correlation rules, alerting, dashboards, threat intelligence enrichment and case management. Every SIEM aggregates logs, but an aggregation platform without detection rules helps only after someone already knows where to look. Automated response to what the SIEM finds is the job of SOAR.
Log aggregation on the CompTIA Security+ exam
Log aggregation is one of the monitoring activities in SY0-801 objective 4.4. Scenario wording such as "the attacker cleared the logs on the compromised server," "analysts must search logs from hundreds of systems in one place" or "collect firewall, server and cloud logs centrally" points to aggregation to a central collector or SIEM. Distractors include enlarging the local log, which does not help once an attacker clears it; archiving, which concerns how long logs are kept rather than where they are gathered; and alerting, which acts on data that aggregation has to deliver first.