In computer architecture, a cache is a small, fast memory that holds copies of a subset of the contents of a larger, slower one. It is indexed by address; its contents are duplicates of data that exists somewhere else; and it is invisible to the program except through timing, since a load returns the same value whether it hits or misses. (Whether that invisibility makes a cache an abstraction, rather than merely an optimization, is a question I have taken up before.) Caches work because programs do not touch memory at random: locations that have just been referenced are likely to be referenced again soon (temporal locality), and locations near one just referenced tend to be referenced next (spatial locality). Those two regularities enable a small memory to stand in for a large one much of the time, improving the effective performance of memory in both dimensions: lower latency, since a cache hit never reaches DRAM, and higher bandwidth, since the cache can sustain more transfers per cycle than memory. In 1965, Maurice Wilkes described the idea under the name slave memory, and IBM shipped the first commercial implementation in the System/360 Model 85, whose designers called it a high-speed buffer until an editor at the IBM Systems Journal suggested a shorter word1.
Modern CPUs are full of structures that resemble caches (small, fast, associative, tagged, evicting their least recently used entries under pressure) but that hold something other than a faithful copy of memory:
- A translation lookaside buffer or TLB holds the result of a page table walk.
- A branch target buffer or BTB holds the recent history of where branches went.
- A store buffer holds data that the program has written, but that memory has not yet received.
- A decoded µop cache holds the output of the instruction decoder.
- A prefetcher’s tables hold the distances between successive addresses touched by individual load instructions.
- The branch predictor remembers, for each - REP MOVSBin a program, whether its copies tend to be short or long, so that the microcode can choose a strategy before the length is known.
- A snoop filter holds records of which cores’ caches hold which lines2.
- A load value predictor holds the values that individual loads returned in the past, and offers them as guesses about what those loads will return next.
There doesn’t seem to be an industry consensus on what to call these hardware constructs. One senior CPU architect I know calls them widgets. This article coins the term covert caches, because they are designed to deliver performance benefits while being unobtrusive to the point of being difficult for software to detect. The term also deliberately echoes the security literature’s covert channel, a path by which information moves between two parties that are not supposed to be communicating. Finalizing the metaphor, many security exploits over the last decade have turned covert caches into covert channels, and a sophisticated adversary can use them to read secrets out of a process, a kernel, or a hypervisor that was supposed to be secure.
Most covert caches are memoization tables for a function that is too expensive to evaluate every time its result is needed, added because a CPU designer noticed that the function’s inputs change far less frequently than its output is consumed. The rest hold state that the machine would otherwise have to wait for: data on its way to memory, or the whereabouts of data already sitting in some core’s cache.
Spoiler alert: covert caches fall into two families, separated not by what they hold, but by whether they are allowed to be wrong. A structure that must be correct, like the TLB, needs an invalidation protocol to keep it consistent with the state it summarizes, and on most architectures, the protocol is implemented in software. A structure that is permitted to be wrong, like the BTB, needs no invalidation protocol, because an incorrect answer only costs time. Rather than being kept consistent in advance, it is updated after the fact: if a branch is mispredicted, the CPU discards the results of every instruction executed on the predicted path, resumes on the correct path, and updates the entry3.
Like caches themselves, covert caches also are excellent examples of innovations that transparently benefited end customers, as described in an earlier article. They require no instruction set extension4, no recompile, and no new API; every program benefits from them without knowing they exist, which is exactly why architects have kept adding them, and also why software can be caught flat-footed when they misbehave.
The Two Families
structure what it memoizes wrong? kept consistent by what went wrong
------------------ -------------------- ------ ------------------ --------------------
TLB, paging- page table walks no software on x86 TLBleed
structure caches (shootdowns);
hardware broadcast
on ARM
µop cache, loop instruction decode no hardware on x86; JCC erratum, SKL150
stream detector software on ARM (both fixed by
disabling)
Snoop filter which cores hold no* hardware back-invalidations;
which lines inclusive-LLC
attacks
Store buffer stores memory hasn't no hardware —
yet received
Fast strings whether each rep yes nobody —
(ERMSB, FSRM) movsb tends to copy
a little or a lot
BTB, return stack where branches went yes nobody Spectre v2,
buffer Retbleed, BHI
Stride and stream per-instruction and yes nobody wasted bandwidth
prefetchers per-page access
patterns
Data memory- which values are yes nobody GoFetch
dependent pointers
prefetcher
Memory dependence which loads alias yes nobody Spectre v4
predictor older stores
Load address and what a load will yes nobody SLAP, FLOP
value predictors read, and return
* pessimistically only: recording a line not held by any core costs an unnecessary snoop;
failing to record a line held by a core breaks coherence.Finding Them: Performance Counters
Every structure in the table is invisible in the instruction set, and every one of them has a capacity that determines where a program falls off a performance cliff: the TLB’s reach, the µop cache’s size and alignment rules, the number of branches the BTB can hold, the number of streams a prefetcher can track. For unprivileged application software, the only window a programmer has onto them is the performance counters, and the only reliable documentation is often the optimization manual, the errata, or a paper by someone who reverse-engineered the structure from its timing5. The most valuable single work in that last category is Agner Fog’s The microarchitecture of Intel, AMD, and VIA CPUs, which he has maintained since 1996, and which describes, core by core, the sizes and behaviors of the µop caches, loop buffers, branch predictors, and store-forwarding rules that the vendors’ manuals describe only in part6.
The place to start is at the top. The top-down method developed at Intel by Ahmad Yasin7 classifies every issue slot in the pipeline as either retiring useful work or lost to one of three causes: the front end (fetch, decode, the µop cache, and the instruction TLB), bad speculation (mispredicted branches and machine clears), or the back end (the data caches, the data TLB, and the execution units). Each category is then subdivided, level by level, until the losses are attributed to specific structures, and several of Intel’s deeper metrics are named for the structures in the table (tma_dtlb_load, tma_dsb_switches, tma_store_fwd_blk). On Linux, perf stat --topdown reports the first level on Intel cores, Intel’s toplev tool (part of pmu-tools) descends the hierarchy, perf stat -M computes any metric by name, and AMD’s Zen 4 and later cores offer the equivalent first two levels as the PipelineL1 and PipelineL2 metric groups, since the --topdown option itself works only on Intel.
The top-down breakdown determines which family of structures to suspect before any individual event is consulted, which matters, because the individual events are numerous, differ between vendors, and are renamed from one generation to the next. Each article in this series gives the events for its own structure, on Intel and on AMD, along with the drift between generations and the signals worth watching outside the PMU, of which the TLB’s shootdown interrupts and perf c2c‘s report of contended cache lines are the two most useful. Those names are taken from the event lists that ship with the Linux kernel’s perf tool, which are the most convenient authoritative source for them. perf list on the machine being measured is the final word on which events that core implements, and what they are called there.
Working Around Them
Once the counters have implicated one of these covert caches in a performance problem, the possible remedies are as diverse as the structures themselves: huge pages for a TLB that cannot reach the working set, a different data layout for a prefetcher that cannot learn the pattern, padding for a cache line that two cores are fighting over, an offset for two buffers whose addresses alias. Each article in this series includes remedies for the structure: first the software changes, then the vendor and operating system controls that disable or restrict the structure. The two families are remedied differently: a structure that must be correct is given less to do, and a structure that is allowed to be wrong is given something it can predict.
Conclusion
The covert caches that are allowed to be wrong were designed under a threat model in which the only adversary was the workload itself, and in which an occasional incorrect answer from an untagged table cost nothing more than a few cycles. The decade since Spectre has been spent retrofitting them with the machinery their designers deliberately left out (privilege tags, flushes on context switch, bits that switch them off for code that handles secrets), converting them, one mitigation at a time, into something closer to the first family, and paying for the conversion in the performance they were built to deliver.
Phil Karlton’s joke that there are only two hard things in computer science, cache invalidation and naming things8, usually is told about software. Covert caches embody both problems at once: we do not call them caches, and for half of them, we decided not to invalidate them at all.
Further Reading
- Agner Fog, The microarchitecture of Intel, AMD, and VIA CPUs — Core by core, the sizes and behaviors of the structures in this series, maintained since 1996 and still the only place most of them are described at all.
- Ahmad Yasin, A Top-Down Method for Performance Analysis and Counters Architecture — The method for deciding which structure to suspect before consulting any individual event.
M. V. Wilkes, "Slave Memories and Dynamic Storage Allocation," IEEE Transactions on Electronic Computers, 1965. The Model 85's designers described their buffer in the IBM Systems Journal in 1968, and the journal's editor, Lyle R. Johnson, is generally credited with proposing cache in place of high-speed buffer. The word comes from the French cacher, to hide, which is apt: a cache that works is one the program cannot tell is there.
A snoop is a request to another core's cache, asking whether it holds a line and telling it to give the line up or discard it. Keeping the cores' caches coherent means that when one core writes a line, every other core holding a copy must be told, and broadcasting that request to every core scales poorly. The snoop filter exists so that the requests go only where they are needed.
A structure that is never obliged to be right needs no coherence protocol, no precise tags, and no flushing on a context switch. That efficiency turned out to be the source of a significant percentage of the last decade's security vulnerabilities.
The transparency is to the application, not to the whole software stack. New hardware often arrives with privileged instructions and model-specific registers that an operating system has to adopt before a covert cache delivers what it promises: AMD's INVLPG and Intel's Remote Action Request broadcast the TLB invalidations that used to require an interrupt to every core; PCIDs sat mostly unused in Linux until Meltdown made an untagged TLB too expensive to keep; huge pages require the kernel to find physically contiguous memory, and ARM's contiguous bit needs it to mark the result; and fast strings needed a CPUID bit, glibc's runtime dispatch, and the kernel's alternatives patching before much software would use them. What the body claims is that the application needs none of this.
None of that reverse engineering needs privilege. Ordinary user-mode code allocates memory in a pattern chosen to provoke one structure, times the accesses, and reads the structure's geometry off the points where the timing jumps. The TLB has been mapped exactly this way: to recover "the hash function and the size of linearly-mapped TLBs," the TLBleed authors mapped a large set of test pages and watched which of them evicted each other, arriving at a description of how a virtual address selects a TLB set on cores whose manuals supply no such detail. Moritz Lipp and his coauthors did the same for the µtag hash behind AMD's L1 way predictor. The technique is also what makes these structures dangerous, since a program that can measure a covert cache precisely enough to describe it can usually also use it as a channel: both of those papers are attack papers, and the reverse engineering is their first section, not their conclusion.
Agner Fog, "The microarchitecture of Intel, AMD, and VIA CPUs: An optimization guide for assembly programmers and compiler makers," last updated May 2026. It is the third of five manuals at agner.org/optimize; the fourth, a table of instruction latencies and throughputs, is its natural companion.