Eric Wang
RSS

Writing

A low-latency Linux host tuning checklist

/13 min read/Reliability/#linux/#performance/#latency/#kernel/#aws/#tuning

Most “low-latency tuning” advice is a list of kernel parameters with no explanation. Applied blindly, it can make a host slower, hotter and harder to support. This post is the checklist I would hand to an engineer setting up low-latency trading hosts: for each item, why it matters, how to check the current state, how to change it, and what it costs.

One thing I have found over the years: Linux kernel tuning is essentially the same in the cloud and in the data centre. The kernel does not care where it runs. What changes is how much of the machine underneath you are allowed to control. I cover the AWS EC2 differences in their own section.

Measure first

Latency tuning is about the tail, not the average. A host with a good median and occasional multi-millisecond stalls is worse for a latency-sensitive workload than a host that is slightly slower but consistent. So before changing anything:

  1. Define the metric. Usually a percentile (p99, p99.9, max) of something the application cares about, measured by the application itself. System tools tell you why; the application tells you whether it matters.
  2. Capture a baseline on the untuned host, under realistic load, long enough to catch rare events.
  3. Change one thing at a time and measure again. Keep the ones that help; revert the ones that don’t.

Useful tools:

ToolWhat it shows
cyclictest (rt-tests)Scheduling/wake-up latency per CPU: how late a thread runs after it should have
hwlat tracerLatency caused by hardware/firmware (e.g. SMIs) that the kernel cannot see
perfWhere CPU time goes; scheduler and interrupt events on specific CPUs
mpstat / sar (sysstat)Per-CPU utilisation, including %irq, %soft, %steal
/proc/interrupts, /proc/softirqsWhich CPUs are taking which interrupts

Examples:

# Wake-up latency on CPUs 2-3, one thread per CPU, SCHED_FIFO priority 95, memory locked
sudo cyclictest --mlockall --priority=95 --interval=200 --affinity=2-3 --threads --duration=10m --histogram=200 -q

# Hardware latency detector (needs CONFIG_HWLAT_TRACER)
echo hwlat | sudo tee /sys/kernel/tracing/current_tracer
echo 1 | sudo tee /sys/kernel/tracing/tracing_on
sudo cat /sys/kernel/tracing/trace
echo nop | sudo tee /sys/kernel/tracing/current_tracer   # turn it off afterwards

# Per-CPU breakdown every second; watch %irq, %soft and %steal
mpstat -P ALL 1

# What ran on CPU 3 for 10 seconds
sudo perf record -C 3 -g -- sleep 10 && sudo perf report

Run hwlat only on a test host: it deliberately spins CPUs with interrupts disabled. And keep the baseline results. They are the evidence for every change that follows.

1. CPU isolation and affinity

Why. The scheduler moves threads around and runs other work (kernel threads, timers, other processes) on every CPU. For a latency-critical thread, every one of those interruptions is jitter. Isolation keeps a set of CPUs for the critical work only.

How to check.

cat /proc/cmdline
cat /sys/devices/system/cpu/isolated
cat /sys/devices/system/cpu/nohz_full
taskset -cp <pid>                    # current affinity of a process
ps -eLo pid,tid,psr,comm | awk '$3==3'   # threads currently on CPU 3

How to set.

  • Boot parameters (example: keep CPUs 0-1 for housekeeping, isolate 2-7): isolcpus=managed_irq,domain,2-7 nohz_full=2-7 rcu_nocbs=2-7
    • isolcpus removes the CPUs from general scheduler load balancing.
    • nohz_full stops the periodic scheduler tick on a CPU when it runs a single task (needs CONFIG_NO_HZ_FULL).
    • rcu_nocbs moves RCU callback processing off those CPUs. On RHEL-family systems, grubby --update-kernel=ALL --args="..." applies them; otherwise edit GRUB_CMDLINE_LINUX and regenerate the GRUB config. A reboot is required.
  • Pin the process: taskset -c 4 ./app, or taskset -cp 4 <pid> for a running one. Applications can also set affinity per thread with sched_setaffinity / pthread_setaffinity_np.
  • cset shield (cpuset) is a runtime alternative that does not need a reboot, but the tool is old and not packaged everywhere. On systemd hosts, AllowedCPUs= / CPUAffinity= and cgroup v2 cpusets do the same job.
  • Red Hat’s tuned has a cpu-partitioning profile that sets these consistently from one variable (isolated_cores= in /etc/tuned/cpu-partitioning-variables.conf).

Trade-off. Isolated CPUs are not load-balanced: if you forget to pin a thread there, nothing will run there, and if you pin two busy threads to one isolated CPU they will fight with no help from the scheduler. You also have fewer CPUs for everything else. nohz_full makes kernel entry/exit slightly more expensive, and the kernel docs note it can reduce throughput. Always keep at least one housekeeping CPU per NUMA node.

2. NUMA locality

Why. On multi-socket hosts, memory and PCIe devices are attached to a specific socket. A thread on socket 0 reading memory or a NIC on socket 1 crosses the interconnect on every access, which adds latency and variance.

How to check.

numactl --hardware
lscpu | grep -i numa
cat /sys/class/net/eth0/device/numa_node      # which node the NIC is on (-1 = unknown/single node)
numastat -p <pid>                             # where a process's memory actually is

How to set.

numactl --cpunodebind=0 --membind=0 ./app

Put the critical threads, their memory and the NIC they use on the same node. Consider disabling automatic NUMA balancing (sysctl kernel.numa_balancing=0) on hosts where you place things by hand, because it migrates pages in the background.

Trade-off. --membind fails allocations rather than spilling to another node when that node is full. Hand placement also means you own capacity planning per node.

3. HugePages and Transparent Huge Pages (THP)

Why. Larger pages mean fewer TLB misses for programs with large working sets. But THP’s background work (khugepaged compaction, and on-fault compaction) can cause unpredictable stalls.

How to check.

grep -i huge /proc/meminfo
cat /sys/kernel/mm/transparent_hugepage/enabled
cat /sys/kernel/mm/transparent_hugepage/defrag

How to set.

  • Explicit HugePages, reserved up front: vm.nr_hugepages=<n> via sysctl, or at boot with hugepagesz=1G hugepages=<n> (1 GiB pages are most reliably reserved at boot). Applications use them via hugetlbfs or mmap(MAP_HUGETLB).
  • THP: set enabled to madvise (only regions that ask for it) or never, at runtime by writing to the sysfs file, or at boot with transparent_hugepage=madvise|never.

Trade-off. Reserved HugePages are taken away from the rest of the system whether used or not. Turning THP off can cost throughput for workloads that benefited from it. Measure with your application.

4. IRQ affinity and irqbalance

Why. Device interrupts land on some CPU. If that CPU is running your critical thread, the thread stops while the interrupt and its softirq run.

How to check.

cat /proc/interrupts             # counts per CPU, per IRQ
cat /proc/irq/<n>/smp_affinity_list
systemctl status irqbalance

How to set.

  • Steer IRQs to housekeeping CPUs: echo 0-1 | sudo tee /proc/irq/<n>/smp_affinity_list.
  • irqbalance will rewrite these unless told not to. Either stop and disable it, or keep it and exclude the isolated CPUs (IRQBALANCE_BANNED_CPULIST in its config on recent versions; managed_irq in isolcpus covers managed interrupts).
  • For NIC queues, you may want a queue’s IRQ on a specific core near the consumer thread (same core or a sibling). Decide deliberately rather than leaving it to defaults. See RSS/RPS/XPS in the kernel’s network scaling doc.

Trade-off. Manual affinity is lost on driver reload or device re-enumeration unless it is applied by a service at boot. Concentrating interrupts on few CPUs can overload them under load.

5. CPU frequency and C-states

Why. Idle cores drop into deep C-states to save power, and waking takes time. Frequency scaling means a core may be running slow when work arrives.

How to check.

cpupower frequency-info
cpupower idle-info
cat /sys/devices/system/cpu/cpu0/cpuidle/state*/name
turbostat --quiet sleep 5        # actual frequency and C-state residency (x86)
tuned-adm active

How to set.

  • tuned-adm profile latency-performance (or network-latency, which builds on it) sets the performance governor and limits C-states through the PM QoS interface.
  • Boot parameters: intel_idle.max_cstate=<n> limits the intel_idle driver; processor.max_cstate=<n> limits the ACPI idle driver; idle=poll stops idling altogether.
  • BIOS settings (on bare metal): C-states, P-states, turbo, power profile. Check the vendor’s low-latency guide.

Trade-off. More power and heat. Fewer idle cores also means less turbo headroom for the others. idle=poll keeps every core at 100% busy and can reduce single-core performance, and on hyper-threaded cores it steals from the sibling. Prefer limiting to a shallow C-state over idle=poll, and measure both.

6. NIC settings

Why. NICs batch work to save CPU: they delay interrupts (coalescing) and buffer packets (ring buffers). Both trade latency for throughput.

How to check.

ethtool -i eth0          # driver and version
ethtool -c eth0          # interrupt coalescing
ethtool -g eth0          # ring buffer sizes
ethtool -l eth0          # number of queues (channels)
ethtool -S eth0          # driver statistics, including drops
sysctl net.core.busy_read net.core.busy_poll

How to set.

  • Coalescing: ethtool -C eth0 adaptive-rx off rx-usecs 0 tx-usecs 0 delivers interrupts immediately.
  • Rings: ethtool -G eth0 rx <n> tx <n>. Bigger rings absorb bursts (fewer drops); smaller rings can keep data warmer in cache. Watch drop counters either way.
  • Busy polling: sysctl -w net.core.busy_read=50 net.core.busy_poll=50 (microseconds), or per socket with SO_BUSY_POLL. The socket spins on the device queue instead of waiting for an interrupt.

Trade-off. Zero coalescing raises interrupt rate and CPU use. Busy polling burns CPU in a tight loop. Both are worth it only on cores dedicated to that work.

Beyond host tuning: kernel bypass. When the kernel network stack itself is the bottleneck, the next step is kernel bypass: frameworks such as DPDK, or Solarflare/AMD Onload and ef_vi, let the application read and write packets directly from the NIC without going through the kernel. That can remove many microseconds, but it is not a host setting. The application has to be written or adapted for it (or, with Onload, run under its socket-acceleration library), and it usually needs specific NICs. That makes it an application architecture decision, so it is out of scope for this checklist.

7. Timers and other kernel noise

The idea in plain terms. Isolating a core (section 1) keeps other programs off it. It does not keep the kernel off it. Even on an isolated core, the kernel still stops by from time to time to do its own housekeeping:

  • The scheduler tick. By default every CPU gets a timer interrupt hundreds to a thousand times a second (depending on CONFIG_HZ), so the kernel can check whether to switch tasks.
  • RCU callbacks. RCU is a kernel mechanism for safely freeing shared data. The clean-up work it queues runs on each CPU.
  • kworkers and other per-CPU kernel threads. Background jobs such as workqueue items, vmstat updates and watchdogs.

Each interruption typically costs only microseconds, so it barely shows in average latency. But your critical thread is paused while it happens, and that is exactly what shows up as occasional spikes at p99.9 and max. The fix is to tell the kernel to do this work on the housekeeping CPUs instead: nohz_full stops the tick on a core running a single task, rcu_nocbs moves RCU callbacks elsewhere, irqaffinity sets the default CPUs for interrupts, and the steps below move the remaining background work.

How to check.

sudo perf stat -e 'irq_vectors:local_timer_entry' -a -A sleep 10   # timer interrupts per CPU
cat /proc/softirqs
cat /sys/devices/virtual/workqueue/cpumask

How to set. Follow the kernel’s per-CPU kthreads guide. Common steps:

  • nohz_full and rcu_nocbs (section 1), plus irqaffinity=<housekeeping CPUs> so interrupts default to housekeeping cores.
  • Restrict unbound workqueues to housekeeping CPUs: write a CPU mask to /sys/devices/virtual/workqueue/cpumask.
  • sysctl vm.stat_interval=<seconds> to update vmstat less often.
  • Consider nowatchdog / nosoftlockup on dedicated hosts, knowing you lose that detection.
  • Avoid running perf, monitoring agents or cron jobs on isolated CPUs.

Trade-off. Each item removes a safety net or makes something less fresh. Document every one.

8. Swap and memory

Why. A page fault that has to go to disk is a disaster for latency. So is direct reclaim or compaction in the allocation path.

How to check.

swapon --show
sysctl vm.swappiness vm.min_free_kbytes
vmstat 1             # si/so columns: swap in/out
sar -B 1             # paging activity, major faults

How to set.

  • Disable swap on these hosts, or set vm.swappiness low.
  • In the application: mlockall(MCL_CURRENT | MCL_FUTURE) and pre-fault memory at startup so the hot path never takes a page fault. Raise memlock limits accordingly.
  • Size the host so it never comes close to memory pressure.

Trade-off. Without swap, memory exhaustion means the OOM killer rather than slowdown. That is usually the right choice for these hosts, but alerting on memory has to be good.

On AWS EC2: what you can and cannot control

The tuning above is the same kernel work in the cloud. The difference is the layer below.

  • Virtualised Nitro instances. You get vCPUs, not physical cores, and you cannot change BIOS settings or see SMIs. A small amount of hypervisor overhead is outside your control. Watch %steal in mpstat. CPU isolation, affinity, IRQ steering, THP, swap and sysctls all work as normal inside the guest.
  • C-states and P-states. AWS documents that only certain instance types (typically larger sizes that own a whole socket, and metal) let the OS control C-states and P-states. On others, the guest settings have no effect. Check the “processor state control” page for your instance type.
  • NUMA. Larger instances expose the host’s NUMA topology; check with numactl --hardware rather than assuming.
  • Bare metal (*.metal). Full access to the processor features the hypervisor would otherwise hide, which is closest to a data-centre host, but the BIOS still belongs to AWS.
  • ENA. The Elastic Network Adapter is the NIC you get on Nitro. ethtool -c/-g/-l work; AWS documents the default coalescing values and how to disable adaptive moderation, and the ENA driver repository has a best-practices guide. Keep the driver up to date.
  • Placement. Network latency between instances depends a lot on physical placement. Cluster placement groups keep instances close inside one Availability Zone.
  • No two instances are identical. Measure on the instance type and size you will run, not on a smaller one to save money.

Keep it documented and reproducible

A tuned host that nobody can rebuild is a liability. The person who tuned it leaves, a kernel update changes a default, or a new host is built from an old image, and the latency is back with no obvious cause.

  • Put every setting in code. Boot parameters, sysctls, tuned profiles, IRQ affinity scripts, irqbalance config, systemd unit CPU settings. I use Ansible for this: one role per host type that templates the GRUB command line, sysctls, the tuned profile and the IRQ affinity service, so a rebuilt host comes out identical. Whatever tool you use, no hand edits on production.
  • Record the reason next to the setting. One line per setting: what it does, why it’s there, and the measurement that justified it.
  • Verify on boot. A small check that compares /proc/cmdline, isolated CPUs, IRQ affinity, governor and NIC settings with what is expected, and alerts on drift.
  • Re-measure after changes. Kernel upgrades, new instance types and new driver versions all deserve a fresh cyclictest and application baseline.
  • Keep the baselines. Old measurements are how you know whether something got worse.

Summary checklist

  • Latency metric defined; baseline captured under load
  • Housekeeping vs isolated CPUs decided; isolcpus / nohz_full / rcu_nocbs set; critical threads pinned
  • Threads, memory and NIC on the same NUMA node; automatic NUMA balancing reviewed
  • HugePages reserved if needed; THP set to madvise or never
  • IRQs off isolated CPUs; irqbalance disabled or configured
  • Governor and C-states set (and confirmed actually controllable on this platform)
  • NIC coalescing, rings and busy polling chosen deliberately
  • Remaining kernel noise reviewed (workqueues, vmstat, watchdogs)
  • Swap off or minimised; application memory locked and pre-faulted
  • Everything in config management, verified on boot, re-measured after change

References