
du on /var/log, and see plenty of room. Yet the node says DiskPressure=True.
That's because DiskPressure is a kubelet protection signal. It tracks available bytes and inodes on the filesystems kubelet monitors, and it isn't the same thing as a full persistent volume. This article covers what fills nodes, what happens when pressure persists, and the controls that keep it from happening.
Key Takeaways
- DiskPressure means a monitored nodefs or imagefs crossed a byte or inode eviction threshold.
- Logs, images, ephemeral storage, inode exhaustion, and mount layout cause most cases.
- Kubelet reclaims first, then evicts pods and blocks new scheduling.
- Prevention combines log rotation, ephemeral-storage limits, image GC, and trend alerts.
Common Causes of Kubernetes Disk Pressure
Kubelet evaluates available bytes and inodes on the filesystems used for node data, images, logs, and ephemeral storage. According to the Kubernetes node-pressure documentation, upstream hard defaults are:
| Signal | Hard threshold |
|---|---|
nodefs.available |
<10% |
imagefs.available |
<15% |
nodefs.inodesFree |
<5% (Linux) |
imagefs.inodesFree |
<5% (Linux) |
These are upstream defaults. Managed services may configure them differently. The symptom is low storage, but the source could be a workload, the runtime, a log collector, or the filesystem layout.
Application and container logs
Kubelet rotates runtime-written stdout and stderr logs. In v1.36 the defaults are containerLogMaxSize of 10Mi and containerLogMaxFiles of 5. Logs that applications write to their own files are not rotated by kubelet.
Typical scenarios:
- Debug logging left on in production
- A crash-looping service producing repetitive errors
- A log collector holding rotated files open
Container images and runtime artifacts
Large layers, frequent deployments, writable container layers, and incomplete garbage collection all consume image filesystem space while active workloads look normal.
Storage paths differ by runtime:
- containerd:
/var/lib/containerd - CRI-O:
/var/lib/containers/storage - Docker Engine:
/var/lib/dockerfor the classic store, or/var/lib/containerdwith the containerd image store
Confirm the active runtime before deleting anything. Kubernetes removed dockershim in v1.24, so don't assume Docker paths on modern nodes.
Ephemeral storage and local workload data
Disk-backed emptyDir volumes, temp files, package caches, spill files, and build artifacts draw from node ephemeral storage. A persistent volume on external storage is not the same thing. A local PersistentVolume is a separate concept too.
Missing requests and limits let one pod or job use more local space than the scheduler assumed. An emptyDir sizeLimit helps, but another consumer can still exhaust shared storage first.
Inodes, filesystem layout, and deleted files
Many small files can exhaust inodes while byte usage looks fine. Separate nodefs and imagefs mounts can also hide trouble: one path looks healthy while another crosses a signal.
df and du may not reconcile for several reasons:
- A process holds a deleted file open, so space isn't freed
- Mount changes alter what you can see
- Kubelet's directory scans miss deleted-but-open files, a blind spot the ephemeral storage docs acknowledge
Check df -i and look for processes holding deleted files open before dismissing an alert.
What Happens If Kubernetes Disk Pressure Is Ignored
Kubelet tries node-level reclamation first. With a shared nodefs, it garbage-collects dead pods and containers, then unused images. If that isn't enough, it evicts end-user pods.

- Soft thresholds wait for a configured grace period.
- Hard thresholds allow immediate termination.
The control plane also adds a node.kubernetes.io/disk-pressure:NoSchedule taint, so new pods avoid the node. Reclaiming images doesn't necessarily free the filesystem holding logs, so the layout matters.
Workload impact
- Pods end up
Failedwith reasonEvicted - Scheduling and image pulls fail
- Restarts repeat and performance degrades
- Logging components may be unable to write, so you lose observability when you need it most
When byte pressure triggers eviction, order depends on whether usage exceeds requests, then pod priority, then usage relative to requests. QoS class is not the direct criterion.
Wider consequences
Evicted workloads move to other nodes, which then fill faster. Recovery actions become disruptive. A single-node capacity issue becomes a cluster stability incident.
An upstream report on Kubernetes 1.30 (issue #127548) even describes other pods being evicted before the hostPath pod that caused the pressure. The node does not always remove the workload that filled the disk.
Warning Signs You're About to Experience Kubernetes Disk Pressure
No single percentage alert catches everything. Combine node conditions, filesystem trends, workload behavior, and events.
DiskPressure=True or eviction-related node events
Start here:
kubectl get nodes
kubectl describe node <node>
Look for these messages:
KubeletHasDiskPressure/kubelet has disk pressureNodeHasDiskPressureon transition,NodeHasNoDiskPressureon recoveryEvictionThresholdMet:Attempting to reclaim ...The node was low on resource: ephemeral-storage
These pod messages signal limit evictions, not necessarily node-wide pressure:
Container ... exceeded its local ephemeral storage limitPod ephemeral local storage usage exceeds the total limit of containersUsage of EmptyDir volume ... exceeds the limit
Free space or inode availability is falling unexpectedly
Work through this sequence:
- Run
df -handdf -ion every mount, including separate nodefs and imagefs. - Use
duon log, kubelet, runtime, and temp directories. - Check for deleted files still held open.
- Query the kubelet Summary API:
kubectl get --raw "/api/v1/nodes/<node>/proxy/stats/summary".

The Summary API reports pod ephemeral-storage, container rootfs and logs, availableBytes, and inodesFree. That lets you attribute usage to a pod instead of only the node.
Identify the consumer before deleting anything. Preserve logs required by retention or incident policy.
Pods become Evicted, Pending, or repeatedly rescheduled
Correlate pod status and events with the affected node. Inspect these areas to find the trigger:
- Pod description and recent events
- Ephemeral-storage requests and limits
- emptyDir usage and log growth
- Image churn and recent releases
How to Prevent Kubernetes Disk Pressure
Treat prevention as layers: limit what workloads write locally, control what the node retains, observe growth, and keep recovery headroom.
Control container log growth
- Set
containerLogMaxSizeandcontainerLogMaxFilesfor your Kubernetes version and runtime - Lower unnecessary verbosity
- Ship required logs to a central destination with a deliberate retention policy
Do this at cluster buildout, after runtime changes, and whenever a new logging pipeline or noisy workload arrives. Avoid ad hoc cleanup DaemonSets and blind truncation of active logs. Ad hoc cleanup can break collectors, permissions, or retention requirements.
Govern ephemeral storage at the pod and namespace level
- Set realistic ephemeral-storage requests and limits
- Use emptyDir
sizeLimitwhere it fits - Apply
ResourceQuota(requests.ephemeral-storage,limits.ephemeral-storage) andLimitRangedefaults per namespace - Reserve space with
kubeReservedandsystemReserved
Base values on measured behavior, not copied examples. Validate them for batch, AI, CI/CD, and data-processing jobs.
Put storage-heavy workloads on dedicated node pools using taints, tolerations, labels, and node affinity. A toleration alone doesn't pin pods to a pool.
Manage images, runtime data, and node filesystem capacity
Defaults in v1.36 for image garbage collection:
| Setting | Default |
|---|---|
imageGCHighThresholdPercent |
85 |
imageGCLowThresholdPercent |
80 |
imageMinimumGCAge |
2m |
imageMaximumGCAge |
0s (disabled) |
Keep images small, trim unneeded layers, and watch deployment churn. Verify which filesystem kubelet monitors before changing mounts or attaching disks.
Bigger disks add headroom. They don't fix runaway logs, temp files, image retention, or inode exhaustion.
Monitor storage trends and automate a safe response
Alert on:
- The DiskPressure condition
- Available bytes and inodes
- Log growth and image filesystem usage
- Pod ephemeral-storage consumption
- The rate at which free space is falling
Useful metric sources:
- kubelet Summary API
- node_exporter (
node_filesystem_avail_bytes,node_filesystem_files_free) - cAdvisor (
container_fs_usage_bytes,container_fs_inodes_free)
Set thresholds from workload behavior and recovery time, not a universal number.
Build dashboards that name the node, namespace, pod, and deployment behind abnormal growth. Correlate growth spikes with releases, restarts, and batch jobs.
Document a safe response path:
- Confirm the alert and affected scope
- Clean up safely
- Cordon or drain as needed
- Reschedule workloads
- Replace or expand the node
- Run a root-cause review

Tips for Long-Term Prevention and Control
- Routine checks: Review nodefs, imagefs, inode, log, image, and ephemeral-storage trends on a schedule. Test alerts against realistic growth and recovery.
- Operator standards: Train teams on persistent vs. node-local ephemeral storage, require production resource policies, and ban unsafe manual deletion.
- Documentation: Keep a version-aware runbook covering diagnostics, approved cleanup, retention rules, escalation owners, and kubelet or filesystem rollback steps.
- Capacity intelligence: For cloud block storage on nodes and stateful workloads, use Lucidity for real-time volume visibility across AWS, Azure, and Google Cloud, and AutoScaler to expand and shrink disks with zero downtime. It complements local log, image, inode, and ephemeral-storage controls; it doesn't replace them.
Conclusion
DiskPressure has identifiable causes: logs, images, ephemeral data, inode exhaustion, and layout problems. Checking one directory leads to the wrong diagnosis.
Log governance, ephemeral-storage policy, image lifecycle management, proactive monitoring, and a tested runbook prevent most avoidable evictions and scheduling failures.
Frequently Asked Questions
What does DiskPressure=True mean in Kubernetes?
Kubelet detected that available bytes or inodes on a monitored node filesystem crossed an eviction threshold. It may block scheduling and evict pods to protect the node. The disk need not be literally full.
Why does Kubernetes show disk pressure when df -h shows free space?
Another nodefs or imagefs mount may be the pressured one. Inodes may be exhausted, or a process may hold deleted files open. Check df -i and the relevant filesystem.
Which filesystems does kubelet monitor for disk pressure?
Kubelet monitors nodefs, imagefs, and containerfs where supported, discovered through the runtime. It ignores unrelated mounts. Exact paths depend on your distribution and runtime.
How do I find which pod is consuming ephemeral storage?
Query the kubelet Summary API for pod ephemeral-storage and container rootfs and logs. Then use kubectl describe pod and node events. Inspect emptyDir, temp files, and runtime directories on the node.
How can I prevent container logs from filling a Kubernetes node?
Configure kubelet log rotation, keep application log levels sensible, and ship logs centrally with retention controls. Verify your collector doesn't hold rotated files open. Alert on abnormal log growth.
Does increasing node disk size fix Kubernetes disk pressure?
It adds recovery headroom, nothing more. Unbounded logs, image buildup, temp-file growth, inode exhaustion, and bad filesystem configuration will keep consuming the extra space.