As we mentioned in the last cmath blog post, MSVC is using LLVM-libc for compile-time evaluation and runtime execution of math functions when
/Zc:cmathis enabled. I invited LLVM-libc contributors Michael Jones and Tue Ly to write a guest blog post about how they’ve achieved what they’ve achieved. This is that blog post! I hope you enjoy it. I learned a lot from reading it, even after having spent a lot of time in this area.– Cody, Microsoft C++ Compiler Frontend Engineer
Hi, I’m Michael Jones, the lead maintainer for LLVM-libc. Cody invited me and our math lead, Tue Ly, to write a guest post about LLVM-libc’s math functions. Tue and I work on LLVM-libc at Google, where we’ve been working on the project for about five years now.
I’m going to talk a bit about the history and philosophy of LLVM-libc, then Tue will do a deeper dive into how and why our math functions are so good. I’m really excited for you all to use the code we’ve been working on, and thanks to Cody for putting all of this together!
1. libc as a Library
Generally, implementations of the C standard library (libc) are written for a specific target. They are usually monolithic libraries that are closely tied to the OS interface they’re based on. They also don’t make any effort to keep individual functions separate, which makes them difficult to break into smaller chunks.
In 2019, LLVM-libc was started with a different design philosophy. The focus was on portability, modularity, and writing clean C++ that could be reused. The original design describes this as writing “libc as a library”, meaning that it would be much closer to an idiomatic C/C++ library. From the beginning, there has been a separation between the OS interface layer and the public interface, as well as making each individual function independent. Portability has also involved avoiding assembly except where absolutely necessary, which has the added benefit of enabling the compiler to make deeper optimizations.
Building on the modular design, I started Project Hand-in-Hand which allowed LLVM’s libc++ to use the same float-to-string conversion code as LLVM-libc. Other LLVM projects have also started to integrate LLVM-libc’s code, such as OpenMP using the printf core and compiler-rt using the software floating point support. Soon both clang and MSVC will be using LLVM-libc’s math library for constexpr evaluation, but what makes LLVM-libc’s math suitable for compiler use?
2. libm as an Opportunity
I feel like I know more than the average programmer about floating point numbers, but I would not consider myself an expert. Tue Ly is definitely an expert, with the experience and credentials to prove it. He was the one who designed LLVM-libc’s math library from the ground up, and it was his idea to make it correctly rounded for all rounding modes. This turned out to be hugely important to Google for much the same reason it’s important to Microsoft: different versions of your math library returning different results can have disastrous consequences when working with heterogeneous distributed systems. I’ll let Tue go into the details of exactly how the math functions are still performant while being so accurate, as well as the path to get there.
3. The “Dark Magic” of libm and Why Consistency Demands Correct Rounding
Thanks, Michael! And thanks to Cody for inviting us to contribute to this series.
In the previous blog post, Cody laid out the four dimensions users care about when it comes to math libraries: Stability, Accuracy, Consistency, and Performance (or, in Cody’s words, “Numbers go brrr”).
Floating-point math in software is often treated like dark magic. When developers see different results across platforms or compilers, they often shrug and say, “Well, floating-point is inexact.” But inexact representation does not mean mathematical operations have to be non-deterministic.
Traditional math libraries (libm) frequently produce different results across platforms and versions. In production systems, these differences cause real issues:
- Instruction-Level Divergence: When hardware designers try to speed up math operations by implementing them directly in silicon (like fast reciprocal square roots or transcendentals), different CPU vendors often make different trade-offs. As a result, the exact same C++ code running on Intel, AMD, ARM, or a GPU can produce slightly different numbers.
- The Trap of Bug-for-Bug Compatibility: Once an inaccurate math routine ships in a major platform or OS, downstream software inevitably starts relying on those exact bits. A classic example is the legacy x87 hardware transcendental instructions (FSIN,FCOS):- Intel’s original hardware implementation used a truncated 66-bit approximation of π for argument reduction. As Bruce Dawson documented in Intel Underestimates Error Bounds by 1.3 quintillion, for certain inputs this causes huge errors, producing inaccuracies of up to ULPs (Units in the Last Place)!
- To preserve bug-for-bug compatibility with software expecting Intel x87 outputs, AMD adopted the same 66-bit reduction behavior in their x87 instruction set, locking the x86 ecosystem into the same legacy behavior.
- On top of that, because these legacy instructions are microcoded in modern processors, they are actually much slower than optimized software implementations using modern range-reduction algorithms like Payne-Hanek.
- Breaking Golden Tests: Golden tests check that a specific input gives a consistent output, usually by comparing against a fixed “golden” result. These are common for image processing pipelines where stability of results is important. Even in software libraries, if a library only guarantees a loose error bound, internal optimizations or bug fixes can shift output bits. A tiny 1-ULP shift in std::sinorstd::powcan invalidate hundreds of golden tests requiring engineering teams to spend time updating test baselines.
The Limits of “Flag-Based” and Environment-Freezing Workarounds
Developers have dealt with floating-point differences across platforms for a long time, as seen across game development, physics simulations, and language standardization:
- Game Engines & Lockstep Simulations: Analyses by Bruce Dawson (Random ASCII, including articles on intermediate precision), Glenn Fiedler (Gaffer on Games), and Forrest Smith (Planetary Annihilation) show how differences in instructions, compiler optimizations, and math libraries routinely desynchronize multiplayer physics engines.
- Custom Math Workarounds: When building Factorio, Wube Software found that standard math functions (sin,cos) returned slightly different results between Windows, Linux, and macOS. Without a correctly rounded math library available, they had to write their own custom software trigonometry routines to keep multiplayer games in sync. Similarly, open-source RTS games bundled custom math libraries like STREFLOP (Spring RTS / Beyond All Reason) to get identical calculations across operating systems.
- C++ Standardization Proposals: The ISO C++ Committee is actively discussing these reproducibility issues across compilers and targets in proposals like P3375 (“Reproducible floating-point results”, Davidson et al.).
However, all of these traditional workarounds share an inherent limitation: while compiler flags and hardware normalization can achieve reproducibility for basic arithmetic operations (, −, ×, /, √), they fall short for transcendental functions (sin, cos, exp, log, etc.), which form a substantial part of standard math.h.
Basic arithmetic operations are strictly specified and hardware-mandated to be correctly rounded under IEEE-754. But transcendental functions in standard math.h are almost always software approximations where IEEE-754 has historically not mandated bit-level exactness. Consequently, their output bits are simply an artifact of a specific library’s polynomial approximations, range-reduction splits, or lookup table values.
The moment a library maintainer updates a function to make it faster, reduce binary size, or vectorize it, the least significant bits will inevitably shift for many inputs.
Freezing compiler flags or using custom lookup tables cannot protect you from future library updates. You are left choosing between never updating dependencies or frequently updating downstream tests.
Why Correct Rounding Solves This
This is where correct rounding fixes the problem.
Under IEEE-754, if a math function is guaranteed to be correctly rounded across all standard rounding modes (round-to-nearest, round-up, round-down, and round-to-zero), the output bit pattern for any given input is mathematically unique, meaning there is only one correct answer.
By solving for Accuracy through correct rounding, you automatically get Stability and Consistency for free. Because the output is mathematically determined rather than an implementation detail, library authors can completely rewrite, optimize, and vectorize their algorithms across future releases without shifting output bits. Your math functions return the exact same bits on x86-64, ARM64, Windows, Linux, across datacenters, across library updates, and, with C++23 (P0533R9) and C++26 (P1383R2) adding constexpr math, between compile-time evaluation and runtime execution.
4. Standing on the Shoulders of Giants: The Road to Production Correct Rounding
In his blog post, Cody joked that writing a complete, accurate math library seemed to require “years of monk-like study.” Cody wasn’t wrong, except it was decades of research across the computer arithmetic community.
The Table-Maker’s Dilemma
For decades, correct rounding for transcendental functions (eˣ, log x, , etc.) was considered too slow for production systems due to the Table-Maker’s Dilemma.
The dilemma is simple to state but hard to solve: when computing an approximation ŷ of a transcendental function y = f(x), how many extra bits of intermediate precision do you need to guarantee that rounding ŷ produces the exact same floating-point value as rounding the exact mathematical value f(x) ?
If the true mathematical result falls very close to a rounding midpoint (for round-to-nearest) or a representable floating-point boundary (for directed rounding), an approximation with standard precision cannot tell whether to round up or down.
The first major attempt to bring correctly rounded transcendentals to production was Abraham Ziv’s work on IBM libultim in the 1990s (later used in glibc). Ziv introduced the multi-stage evaluation strategy: compute a fast approximation; if the result is too close to a rounding boundary (Ziv’s rounding test), fall back to a higher-precision path.
However, because the maximum required precision was unknown at the time, libultim‘s fallback path used arbitrary-precision arithmetic scaling up to 768 bits of precision (via internal helpers like slowpow.c and mppow.c). When programs hit one of these hard-to-round inputs, performance dropped sharply: while a normal fast-path pow took ~70 cycles, the 768-bit fallback took up to 440,000 cycles (as documented in glibc Bug 13932 and Bug 16898) which is a ~6,000× slowdown on a single function call!
These unpredictable slowdowns caused performance issues in production workloads, eventually leading glibc maintainers to remove libultim in favor of faster, non-correctly rounded routines with bounded latency. As a result, correct rounding got a reputation for being “too slow for production.”
Hunting the Worst Cases
To avoid these slowdowns, the arithmetic research community needed to answer a fundamental question: What is the worst-case hard-to-round input across the entire floating-point domain? If you know the worst case, you know the maximum precision ever required, and you can build a bounded, non-allocating fallback path.
Finding these worst cases is very difficult. In double precision, searching the 2⁶⁴ input space by brute force was long considered computationally prohibitive.
The breakthrough began with the foundational work of Vincent Lefèvre and Jean-Michel Muller (Arénaire / AriC at ENS Lyon / CNRS / Inria). In pioneering publications like Toward correctly rounded transcendentals (Lefèvre, Muller, & Tisserand, 1998) and Worst Cases for Correct Rounding of the Elementary Functions in Double Precision (Lefèvre & Muller, 2001), they proved that the worst-case precision required for double-precision functions could be systematically determined. Their work culminated in the release of CR-LIBM, demonstrating for the first time that double-precision elementary functions could be correctly rounded with reasonably bounded intermediate precision.
Over the last several years, this search was completed. Through our close collaboration with Paul Zimmermann and Vincent Lefèvre, culminating in our recent paper Computing hard-to-round cases of sin, cos, tan in double precision (ARITH 2026), we have now systematically searched and proven the worst-case bounds for all standard univariate double-precision math functions.
Most importantly, all of these worst-case inputs and mathematical bounds are openly hosted and maintained in the CORE-MATH project repository (and documented at core-math.gitlabpages.inria.fr), serving as an open, shared ground truth for library implementers worldwide.
Bringing Correct Rounding to Production
Around 2020, when we started architecting the math library for LLVM-libc, several independent developments were converging:
- Proven Mathematical Bounds: Decades of search algorithms had established the hardest-to-round cases, proving that 128-bit intermediate precision is sufficient for most univariate double-precision functions.
- Modern 64-bit Hardware: Modern processors had native hardware Fused Multiply-Add (FMA), wide vector pipelines, and rich register files capable of executing branchless 128-bit integer arithmetic in a handful of cycles.
The realization that the timing was finally right wasn’t unique to LLVM-libc. Around the exact same time, two other major research efforts started with the same objective:
- The CORE-MATH project led by Paul Zimmermann at Inria, focusing on open-source, correctly rounded reference implementations.
- The RLIBM project led by Santosh Nagarakatte at Rutgers University, exploring novel polynomial generation techniques for correct rounding.
From the very beginning, the LLVM-libc math team established an active collaboration with CORE-MATH and RLIBM: discussing algorithm approaches, sharing hard-to-round test vectors, developing parallel implementations, and cross-validating each other’s implementations.
This collaboration helped us verify correctness, optimize performance, and prove that correctly rounded math is fast enough for production use.
5. Under the Hood: Making Numbers Go Brrr with Correct Rounding
When developers hear that LLVM-libc guarantees correct rounding, their immediate reaction is often skepticism: “If you’re checking every last bit against the exact mathematical value, how is your library not slow?”
The answer lies in how we structure our execution pipeline around modern hardware strengths:
When an input comes in, we don’t immediately jump into high-precision math. Instead, our first stage evaluates a fast, carefully tuned Taylor or minimax polynomial approximation using double-double or truncated double-double arithmetic to achieve 70-106 bits of precision, heavily accelerated by native hardware FMA instructions. We break the function’s domain into small intervals such that lookup tables (if needed) comfortably reside in the CPU’s L1 cache.
Once the fast path computes this higher-than-double approximation, we perform Ziv’s rounding test: we check whether the error interval contains a rounding boundary. If the answer is clear (which happens for over 99.99% of all possible inputs), we round to the target precision and return immediately. In terms of latency and throughput, this fast path is very similar to fast, non-correctly rounded math libraries.
What about that rare fraction of a percent (<0.01%) of tricky, hard-to-round inputs where the fast path is inconclusive?
Because the maximum intermediate precision is mathematically proven to never exceed 128 bits, our second stage never needs to invoke an arbitrary-precision library or allocate memory on the heap. Instead, it executes a fixed, branchless 128-bit or 256-bit integer routine with a predictable, bounded cycle count. There are no latency cliffs.
The Freedom to Optimize
Guaranteed correct rounding completely changes the economics of math library development. Historically, maintainers avoided modifying math routines because any tweak, such as changing a polynomial degree or shrinking a lookup table, would shift output bits and break downstream regression tests. With correct rounding, that concern disappears. Because the output is mathematically determined, we are free to explore all kinds of performance improvements: tuning polynomial degrees, optimizing cache footprints, leveraging target-specific hardware instructions (like FMA and SIMD), or rewriting entire fast paths without worrying about breaking downstream software.
Additionally, implementing the entire library in clean, standard C++ gives the compiler optimizers full visibility into the code. Rather than relying on inline assembly or platform-specific constructs that obscure dataflow, modern compilers can freely inline helper functions, schedule instructions to increase instruction-level parallelism, and auto-vectorize loops across different architectures.
Conclusion & Joining the Community
Bringing correctly rounded math into production has been a multi-year effort spanning academic pioneers, open-source maintainers, and industry partners.
Today, software stacks are becoming increasingly complex and distributed. With machine learning models and mixed-precision pipelines already introducing non-determinism, having standard math functions return identical results across platforms and versions helps keep software reliable and predictable.
We are very excited to see MSVC adopting LLVM-libc’s math library for constant evaluation and more in C++23, bringing deterministic floating-point math to all the Windows developers worldwide.
If you’re interested in contributing to LLVM-libc, either for the floating-point math effort or the overall standard library development, we’re always looking for new contributors! You can find more information including how to set up a local build at https://libc.llvm.org/. You can also find the community on the LLVM discord in the #libc channel or join one of our regular public meetings. We hope to see you there!
Michael Jones
Tue Ly