Explore our new Cybersecurity Terms Glossary, explained by our experts.

View the Glossary View the Glossary
PRIVIA

Cybersecurity Terms Glossary

What Is Kubernetes Incident Analysis?

Investigating a Kubernetes compromise means reading the API server audit log, the cluster state, and node-level telemetry, because the workloads themselves are usually gone before anyone looks.

Illustration for the cybersecurity term Kubernetes Incident Analysis
Home / Blog / Cybersecurity Terms / What Is Kubernetes Incident Analysis?
17 November 2026

What Is Kubernetes Incident Analysis?

What Is Kubernetes Incident Analysis?

A Kubernetes cluster is a control loop that keeps replacing things. Pods are rescheduled, nodes are replaced by the autoscaler, and a compromised container may have been deleted and recreated a dozen times before an alert is triaged. Investigating an incident here means working from the records of what the cluster was asked to do, rather than from the artefacts of what it did.

The API Server Audit Log Is the Primary Evidence

Every action in a cluster passes through the API server, which can log each request with the authenticated identity, the source address, the verb, the resource, and the response. Configured at RequestResponse level for sensitive resources, the audit log answers the central questions: which identity created the privileged pod, when the role binding changed, who read the secrets, and from where.

This is also where preparation decides outcomes. Managed services do not all enable audit logging by default, or they retain it briefly, and self-managed clusters need an audit policy written by hand. A cluster without an audit log leaves an investigator reconstructing events from fragments, and the honest finding in that situation is that the question cannot be answered.

Kubernetes events are a weaker substitute. They are useful, but they live in etcd with a default retention of about an hour, so they are gone long before most investigations start unless they are shipped elsewhere.

What Compromise Usually Looks Like

Preserving Evidence Without Killing It

The instinct to delete a suspicious pod destroys everything. A better sequence is to cordon the node so nothing new is scheduled onto it, apply a network policy that isolates the pod rather than terminating it, and set the workload’s replica count so the controller does not immediately recreate what you are examining. Then capture node memory, preserve the container’s writable layer as described in Docker container forensics, export the pod specification and the relevant cluster state, and snapshot any persistent volumes.

Runtime security tooling is what fills the gap for workloads that no longer exist. Falco, Tetragon, and equivalent eBPF-based sensors record process execution, file writes, and network connections per container and ship them off the node immediately, which converts an ephemeral workload into a durable record.

Scope the Blast Radius

One compromised pod is rarely the whole story. Establish which service account the workload used and what that account could reach, whether secrets were mounted into it, whether the node hosted pods from other namespaces or tenants, and whether the cluster’s cloud identity grants access to resources outside it — a node role with broad permissions turns a container compromise into a cloud incident. Then rotate: service account tokens, secrets that were mounted, and any cloud credentials the workload could reach.

What to Enable Before You Need It

Audit logging at a sensible level with off-cluster retention, control plane and node logs shipped to a central store, runtime sensors in at least detect mode, admission control that blocks privileged pods, and image digests recorded at deployment. The NSA and CISA Kubernetes hardening guidance covers the preventative side of the same list. Our cloud security services include assessing whether a cluster could actually be investigated today, which is a different exercise from assessing whether it is hardened.

FAQ – Frequently Asked Questions

Where Is the Most Important Evidence in a Kubernetes Incident?

The API server audit log. Every action in the cluster passes through the API server, so the log records which identity created or modified each resource, when, and from which address. Without it, you are reconstructing events from node logs and container output. Check that it is enabled, set to a useful level, and retained off-cluster before you need it.

Should I Delete a Compromised Pod?

Not as a first step. Deleting it destroys the container’s writable layer, its process memory, and often the only record of what ran. Cordon the node, isolate the pod with a restrictive network policy, and prevent the controller from immediately recreating it. Then capture memory, preserve the filesystem layer, and export the cluster state before removing anything.

How Long Are Kubernetes Events Retained?

About an hour by default, because they are stored in etcd with a short time to live. They are useful during live triage and effectively useless afterwards unless they have been shipped to a central log store. Treat them as a supplement to the audit log rather than a substitute, and configure forwarding if you want them available later.

Does a Container Compromise Mean the Cluster Is Compromised?

Not necessarily, but you have to prove it rather than assume it. Establish what the pod’s service account could access, whether secrets were mounted, whether the container was privileged or mounted host paths, and what cloud permissions the node identity holds. Any of those can turn a single workload compromise into cluster or cloud account compromise.

Related Terms

You May Be Interested In These

What Is Process Doppelganging?

Cybersecurity Terms

17 November 2026

What Is Process Doppelganging?

Read More
What Is a JSON Web Token?

Cybersecurity Terms

17 November 2026

What Is a JSON Web Token?

Read More
What Are Short-Lived Certificates?

Cybersecurity Terms

17 November 2026

What Are Short-Lived Certificates?

Read More