Explore our new Cybersecurity Terms Glossary, explained by our experts.
Cybersecurity Terms Glossary
Log management is the end-to-end handling of machine-generated records: generating them, getting them off the host, parsing them, storing them affordably, and being able to find anything in them later.

What Is Log Management?
Log management covers everything that happens to a record between the moment a system writes it and the moment someone needs it eighteen months later. Generation, collection, transport, parsing, storage, access control, retention, and disposal. It is unglamorous and it determines whether your detection, your investigation, and your regulatory evidence are possible at all.
The distinction still causes procurement mistakes. Log management is about getting records somewhere reliable and finding them again; a SIEM adds correlation, detection content, case management, and analyst workflow on top. Many organisations have bought the second and neglected the first, which is why their detections run over a subset of sources that happened to be onboarded during the project and never revisited.
The economics differ too. SIEM licensing is usually volume-based, which creates pressure to collect less. Log management can be cheap at scale if storage is tiered properly, which is why a common architecture now sends everything to an affordable store and routes only the detection-relevant subset into the analytics platform.
Each stage has a characteristic failure, and all of them are quiet.
| Stage | Typical failure |
|---|---|
| Generation | Auditing never enabled on the source |
| Collection | Agent installed but not reporting |
| Transport | UDP syslog silently dropping under load |
| Parsing | Vendor changes the format in an upgrade |
| Storage | Retention shortened to control cost |
| Access | Nobody can run the query during an incident |
Transport deserves particular attention. Plain syslog over UDP, still the default on a great deal of network equipment, loses records without reporting it. Use TCP with TLS where the source supports it, and where it does not, place a collector close to the device and monitor the volume it receives.
Tooling in common use includes rsyslog and syslog-ng for collection, Fluent Bit and Vector for routing and transformation, Windows Event Forwarding for Windows estates, and OpenTelemetry collectors where the same pipeline carries operational and security data. The NCSC’s Logging Made Easy, retired in 2023 and relaunched by CISA later the same year, remains a reasonable free starting architecture for smaller organisations.
The single most valuable control in log management is detecting silence. Record the expected volume per source per hour and alert when it deviates, which catches an agent that stopped, a firewall rule that blocked a collector, and a disk that filled. Add a synthetic event generated on a schedule and verified at the far end, so you are testing delivery rather than inferring it.
Measure ingestion lag as well. A pipeline that delivers reliably but 40 minutes late has quietly made your correlation rules unworkable and your response slower than the attacker.
Logs are evidence. An attacker with administrative rights will attempt to clear them, which is why Windows event 1102 and the Linux equivalent deserve high-severity rules, and why records must leave the host quickly rather than accumulating locally. Write-once storage for the archive tier, access logging on the log platform itself, and separation of duties so that system administrators cannot alter the records of their own activity are the standard controls.
PCI DSS v4.0 requirement 10 sets the most prescriptive expectations: audit logs for all in-scope system components, protected from modification, retained for 12 months with the most recent three months immediately available. The NCSC Cyber Assessment Framework principle C1.b expects logs to be collected and secured against tampering. And UK GDPR cuts the other way: logs frequently contain personal data, so storage limitation applies, the retention period needs a stated justification, and staff should be told about monitoring through an employee monitoring policy. Keeping everything forever is not a neutral choice.
Those tensions are resolved in retention planning, which is where most of the hard decisions in log management actually get made. At Privia Security, pipeline health is the first thing we check in our SOC maturation services, because detection content built on a broken pipeline is decoration.
Log management collects, transports, parses, stores, and makes records searchable. A SIEM adds correlation across sources, detection content, alerting, and case management on top of that foundation. You can run log management without a SIEM and still investigate effectively; a SIEM without reliable log management produces detections over whichever sources happened to be onboarded, which is usually fewer than anyone assumes.
Identity and authentication systems, endpoint process telemetry, DNS, firewall and proxy records, and the audit trails of your critical business applications. These support the largest number of detections and answer the questions investigations ask most often. Add sources deliberately against detection requirements rather than onboarding whatever is easiest, which is how estates end up collecting volume without visibility.
Baseline the expected volume for each source by hour of day and alert on significant deviation in either direction. Supplement that with synthetic events generated on a schedule and verified at the destination, which tests the whole path rather than inferring it. Silent loss is the normal failure mode in logging, so treat pipeline monitoring as a detection in its own right.
For most organisations yes, because tiered cloud storage makes long retention affordable in a way that on-premises capacity rarely does. Consider where the data resides, since logs contain personal data subject to UK GDPR, and ensure the archive remains accessible during an incident that affects your cloud tenancy. Keeping a copy outside the environment being monitored is sound practice regardless of where the primary store sits.
You May Be Interested In These
We use a few cookies that are needed to make this website work, and they are always on. We would also like to set analytics cookies, to see which pages are useful; functional cookies, so that our live chat works; and advertising cookies, to measure our campaigns. We will not set any of those until you say yes, and you can change your mind at any time from the Cookie Settings link in the footer.
Choose the cookies you are happy for us to use. Strictly necessary cookies are always on; everything else is off until you turn it on.
These cookies keep the website working: they hold your security checks, your form submissions and the choice you make here. The site cannot run without them, so they are always on.
These let our live chat open and remember your conversation, and let the newsletter sign-up work. Switch them off and the chat window will not appear.
These tell us which pages are read and how people find us, so that we can improve the site. The reports are about groups of visitors, never about you personally.
These let us measure our advertising and show you our adverts on other websites, such as LinkedIn, Google and Meta. Switch them off and you will still see adverts, but they will be less relevant.