How to Find the Cause of High Load Average with Low CPU Usage in Linux

Why Load Average Can Be High While the CPU Is Mostly Idle

Load average in Linux does not show CPU utilization as a percentage. It shows the average number of tasks that were running, waiting for CPU time, or in uninterruptible sleep. In the output of uptime, the three values represent averages over 1, 5, and 15 minutes.

uptime
 17:20:41 up 18 days,  4:12,  2 users,  load average: 12.84, 10.21, 8.67

The calculation primarily includes processes in these states:

• R — running on a CPU or waiting to be scheduled;
• D — in uninterruptible sleep, usually waiting for a disk, network, or another kernel operation.

Therefore, a server can show a load average of 20 while CPU utilization remains at 5–10% if many processes are stuck in the D state. The CPU is idle, but the tasks cannot continue until the kernel completes the I/O operation.

Do not compare load average directly with CPU utilization. First determine the number of available CPUs, then check the runnable queue and the number of processes in the D state.

Step 1. Assess the Scale of the Load

Find the number of logical CPUs:

nproc
lscpu | grep '^CPU(s):'

Load average is not normalized by the number of processors. A value of 8 means different things on a 2-core server and a 16-core server. On a system with 16 CPUs, a load of 8 does not by itself indicate CPU overload. On a system with 2 CPUs, the same value requires investigation.

Check the trend:

uptime
cat /proc/loadavg

Example contents of /proc/loadavg:

12.84 10.21 8.67 3/812 24591

The first three numbers are the load averages. The value 3/812 means that three tasks are currently running or ready to run, while the system is tracking 812 tasks in total. The final number is the PID of the most recently created process.

If the first value is noticeably higher than the second and third, the load is increasing. If it is lower, the problem is subsiding. High 15-minute values with a normal current state may be the result of an incident that has already ended.

Step 2. Check CPU Usage, the Run Queue, and Blocked Processes

Use vmstat for an initial check:

vmstat 1

The most important columns are:

• r — tasks waiting for CPU time;
• b — tasks in uninterruptible sleep;
• us and sy — CPU time spent in user mode and kernel mode;
• id — CPU idle time;
• wa — time spent waiting for I/O;
• si and so — pages read from swap and written to swap.

If r consistently exceeds the number of CPUs, there is probably a CPU run queue. If b is high while id remains high, look for blocked operations.

The wa value is useful but is not conclusive evidence. It may remain low even when processes are in the D state, for example when the wait is related to a network file system, a device driver, or a separate queue while other CPUs remain idle.

Step 3. Find Processes in the R and D States

Display the PID, state, elapsed time, command, and wait channel:

ps -eo pid,ppid,user,stat,etime,wchan:32,comm,args --sort=stat

To select only the relevant states:

ps -eo pid,ppid,user,stat,wchan:32,comm,args |
awk '$4 ~ /^[RD]/'

The state may contain additional characters, such as Ds or D<. For diagnostics, the first letter is what matters.

The wchan field shows the kernel function in which the process is sleeping. Typical values may indicate that a process is waiting for a disk, file system, network response, lock, or child-process completion. The function name is not always sufficient for a definitive conclusion, but it helps group blocked tasks.

It is also useful to count processes by state:

ps -eo stat= |
cut -c1 |
sort |
uniq -c |
sort -nr

If the number of D processes is increasing, record their PIDs and commands. A single blocked process usually does not create a high load average. Dozens or hundreds of similar blocked processes indicate a system-wide cause.

Step 4. Check the Storage Subsystem

High load average with low CPU usage is often caused by a slow storage device, exhausted IOPS, high latency, RAID problems, or a saturated block-device queue.

If the sysstat package is installed, run:

iostat -xz 1

Evaluate the values over time:

• r/s and w/s — the number of read and write operations per second;
• r_await and w_await — average read and write latency;
• aqu-sz — average queue length;
• %util — device utilization.

There is no single universally acceptable latency threshold: normal values differ for a local SSD, HDD, network block device, and cloud volume. It is more important to compare the metrics with the normal baseline for the specific server and look for a sudden increase in latency or queue length.

Identify the processes generating I/O:

pidstat -d 1

Also check available space and inodes:

df -h
df -i

A full file system does not always increase load average directly, but it can cause applications to hang, trigger retry loops, produce database errors, and disrupt logging.

Checking Device and File-System Errors

Review kernel messages from the current boot:

journalctl -k -b --no-pager |
grep -Ei 'error|fail|timeout|reset|I/O|blk|nvme|scsi|ata|ext4|xfs|btrfs'

Pay particular attention to messages about timeouts, controller resets, I/O errors, hung tasks, and file systems being remounted read-only.

Do not restart the server before saving the logs if the situation allows you to continue the investigation. After a reboot, some evidence may disappear while the underlying problem remains.

Step 5. Check Network File Systems

NFS, CIFS, and other remote file systems can keep processes in the D state when the storage server is unavailable, routing is broken, or a network request is stuck.

Find network mounts:

findmnt -t nfs,nfs4,cifs,fuse.sshfs

Check which processes are using a suspicious mount point:

fuser -vm /mnt/storage

Commands such as df, ls, or du may themselves hang when accessing an unavailable network resource. Run them with a timeout:

timeout 5s stat /mnt/storage
timeout 5s ls -la /mnt/storage

If the command times out, check storage-server availability, network status, DNS, filtering rules, and NFS or CIFS logs. Forcefully terminating a process in the D state usually does not help: the signal will be handled only after the process returns from the kernel.

Step 6. Check Memory and Swap

Insufficient memory can cause intensive page swapping, I/O stalls, and an increase in load average.

free -h
vmstat 1
swapon --show

Used swap space does not by itself indicate a problem. More important signs are persistently nonzero si and so values in vmstat, degraded performance, and increasing storage latency.

Check for OOM messages:

journalctl -k -b --no-pager |
grep -Ei 'out of memory|oom-killer|killed process'

It is also useful to evaluate resource pressure through PSI:

cat /proc/pressure/cpu
cat /proc/pressure/io
cat /proc/pressure/memory

The some metric shows the percentage of time during which at least some tasks were delayed because of the resource. The full metric shows periods when all non-idle tasks in the relevant group were stalled by that resource at the same time. High io values with low CPU utilization confirm that the system is I/O-bound.

Step 7. Rule Out Container and cgroup Limits

Inside a container, the available CPU may be restricted by a quota even though the system reports a larger number of logical processors. In that case, the task queue grows while overall utilization on the physical host remains low.

For cgroup v2, check the quota:

cat /sys/fs/cgroup/cpu.max
cat /sys/fs/cgroup/cpu.stat

In cpu.max, the value max means that no quota is set. Two numbers specify the allowed CPU time and the period. In cpu.stat, increasing nr_throttled and throttled_usec values indicate that tasks are regularly being limited by the CPU quota.

On a server running systemd, view resource usage by group:

systemd-cgtop

This scenario differs from I/O stalls: processes are more likely to be in the R state than in the D state.

Step 8. Investigate a Specific Stuck Process

For the selected PID, first collect basic information:

PID=1234
cat /proc/$PID/status
cat /proc/$PID/wchan
cat /proc/$PID/io
ls -l /proc/$PID/fd

The kernel stack may reveal where the process is waiting:

cat /proc/$PID/stack

Reading some files in /proc requires root privileges. Interpret the stack together with the application type, file system, and kernel messages.

For a process that is not in uninterruptible sleep, you can briefly trace system calls:

timeout 10s strace -tt -p 1234

If the process is stuck inside a read, write, synchronization, or file-access system call, the trace will show the most recent operation. For a process deep in the D state, no output may appear until the wait ends.

Common Causes and Indicators

Final Checklist

1. Compare load average with the number of available CPUs.
2. Use vmstat 1 to check r, b, wa, si, and so.
3. Find processes in the R and D states.
4. Group blocked processes by command and wchan.
5. Check disk latency, queue length, and active I/O consumers.
6. Review the kernel log for device, file-system, and hung-task errors.
7. Check NFS, CIFS, and other remote mount points.
8. Evaluate memory, swap, and PSI metrics.
9. Check CPU quotas and throttling in cgroups or containers.
10. Save diagnostic data before restarting services or the server.

The main reference point in this investigation is the process state. A large number of tasks in D usually points to storage, a file system, the network, or a driver. A large number of tasks in R indicates a CPU run queue, a cgroup limit, or insufficient compute resources. Load average itself describes the symptom, not the source of the problem.