|<<>>|1 of 344 Show listMobile Mode

Toub’s 224-page tour-de-force on performance in .NET 11

Published by marco on

 The article Performance Improvements in .NET 11 by Stephen Toub (Microsoft Dev Blogs) is the latest in a long line of similar articles for other .NET releases.

See previous coverage in:

Fully half of the 230+ pages is about JIT optimizations (many, though not all, of which are presumably also valid for AOT builds). I’ve collected the parts that I found interesting in several sections below.

“The sections that follow are full of real improvements. A bounds check removed, an allocation that no longer happens, a lock that isn’t taken, a loop that runs in fewer cycles than it did a year ago, a comparison folded to a constant here, a redundant check hoisted out of a loop there, a couple of instructions fused into one, a syscall sidestepped, an array copy handed off to SIMD, and on and on. That’s how real performance work goes, accumulating gain after gain, each compounding on the last, until the whole thing is measurably, provably [faster].”

Escape analysis

“But for other cases where it can’t prove this with static analysis, the JIT turns to profile-guided optimization (PGO). PGO sounds fancy, but it’s conceptually simple. With “tiered compilation”, when a method is first invoked, it can be compiled “just in time” with few-to-no optimizations (this is referred to as Tier 0). The JIT can include in this compilation additional probes (think “printf debugging”) that let it track a bunch of interesting information about the nature of the code, recording what actually happens when it runs: which branches are taken, what are the concrete types that show up at virtual call sites or cast attempts, and so on. If the method is invoked enough or loops enough times, the runtime can ask the JIT to produce a new optimized version (referred to as Tier 1). That compilation can then factor in all of the learnings gathered as part of that profiling.”
// Approximately what the JIT generates
if (animal?.GetType() == typeof(Dog))
{
    ((Dog)animal).Speak();  // devirtualized, inlinable
}
else
{
    animal.Speak(); // original virtual call, hopefully rare
}

“This “guess and verify” pattern, called “guarded devirtualization” (GDV), accounts for many of the biggest throughput wins in real workloads. It’s applicable not only to virtual dispatch but also to interface dispatch […]

““Escape analysis” is the compiler technique that lets us ask whether this object ever “escapes” the current method. If an object reference to a newly allocated object provably doesn’t escape, then the JIT can more efficiently allocate it. It needn’t store it on the GC heap, because nothing could possibly need to reference that object again, so it can instead allocate the object on the stack, making both allocation and cleanup essentially free. Stack allocation is even faster than heap bump-pointer allocation; it’s just decrementing the stack pointer, which is typically already in a register. And more importantly it means zero GC impact, because the stack frame is freed atomically on function return.”

Runtime async

“That’s quite a lot of generated code for three lines of C#. The compiler has to make decisions before the program runs about the state-machine layout, which values might need to survive, how many awaiter fields are required, and how all the suspension points fit into one MoveNext dispatch. The runtime and JIT have optimized the resulting pattern heavily over the years, including combining the task, state machine, continuation, and ExecutionContext into a single allocation, but by the time the JIT sees the IL, the transformation has already happened, leaving it with a very complicated system to try to optimize.

“.NET 11 introduces a new way to split that responsibility, a reimplementation of the async/await infrastructure referred to as “runtime async”. Rather than the C# compiler being responsible for the transformation, the JIT is. The C# compiler emits a much smaller suspension-aware IL contract for each eligible async method and marks the method as async in metadata. The runtime and JIT then do the work that depends on runtime knowledge: creating the externally visible Task or ValueTask, recognizing direct async calls, deciding which values are actually alive at each suspension point, laying out continuation objects, and generating the control flow that suspends and resumes the method. Effectively, the transformation moves from C# to the runtime, where more information is available to optimize it.”

“Runtime async doesn’t make every asynchronous operation allocation-free, though. Rather, it gives the JIT enough information to avoid materializing some task objects that existed only to carry a result from one async method directly into the next. If a consumer stores the task in a collection, manually hooks up a continuation, or otherwise observes the task as an object, that object is still needed. The optimization is about not paying for boundaries that aren’t observably boundaries.”
“Workloads with many layers of small async methods can benefit the most from runtime async, because those layers are exactly where intermediate tasks and state machines often accumulate. Shared framework code, for example, is full of this pattern: a public method validates arguments and awaits a private helper, which awaits a transport helper, which awaits an operating-system operation. Application services similarly compose authentication, retry, logging, serialization, and I/O helpers. Runtime async can make the source-level decomposition cheaper without asking the developer to flatten the code into one giant method in order to avoid “implementation detail” costs.”
“The work is also not only about direct performance improvements but also about improvements to diagnostics and performance tooling that help you to make better use of async in your own code. When an async method suspends, its physical thread stack unwinds. That method’s continuation might later run on a different thread whose physical stack begins in the thread pool, with the methods that led to the original await nowhere to be found. A sampling CPU profiler can see where the processor is spending time, but without additional information, it can’t reliably connect those traces back through the logical async call chain, making it hard to answer questions about what async call paths were actually costing. Profiling tools like the async profiler in Visual Studio have traditionally reconstructed those chains from events emitted by Task‘s infrastructure, but async-heavy applications can generate enormous volumes of those very chatty events. The resulting overhead easily perturbs the workload being measured, making it all but unusable in production.”

Unsafe

“Much of this work wasn’t motivated by contrived benchmarks containing nothing but array indexing as I’m prone to use in these posts. Rather, many of the improvements stemmed from an ongoing audit of unsafe code throughout the .NET libraries, part of a broader effort to improve memory safety in .NET. .NET and C# are memory safe, but as with other memory safe languages like Rust, it provides escape hatches that enable turning off the guardrails provided by the compiler and runtime. This effort is about reducing where and when developers feel compelled to use those escape hatches, since every occurrence is an opportunity for increased risk. Unsafe code was often introduced years earlier to manually avoid bounds checks, typically by walking a buffer with pointers, byrefs, or Unsafe.Add. Sometimes the audit found that the unsafe code was no longer needed and could simply be removed. Sometimes a “safe” rewrite (meaning not using unsafe and friends) was already just as fast or even faster. And sometimes the rewrite exposed an optimization the JIT was missing, in which case the answer was to improve the JIT and then rewrite the library code to use normal, bounds-checked C#.

“[…] If you own code that has hand-written pointer or Unsafe-based loops whose purpose is to avoid bounds checks, it’s worth rewriting them with safe, bounds-checked C# and measuring again on .NET 11. Chances are, you’ll find the gap at this point is either non-existent or small enough that it’s not worth the increased maintenance and risk for managing the safety yourself.”

“The end goal is to make unsafe regions smaller, make their assumptions visible through the call graph, and make them easier for reviewers and tools to find. Pairing that with a JIT that makes idiomatic safe code fast removes a lot of the historical pressure to use unsafe code in the first place.”

Branching

“Branches offer another opportunity for simplification. Modern processors work on several instructions at different stages at the same time. When a processor encounters a conditional branch, it predicts which path will be taken so that it can continue fetching and executing instructions speculatively. A correct prediction hides much of the branch’s cost. A misprediction throws away that speculative work, redirects instruction fetch to the correct path, and refills the processor’s execution pipeline. That can make the predictability of a branch as important as the work in either branch.”

Preserving vector representation

“Once the intrinsic operation has been recognized, the backend still needs to keep it in a useful vector form while assigning registers and selecting instructions. Vector values are structs, and the JIT will often apply “struct promotion,” tracking a struct’s fields as independent locals so that each can be optimized separately. That’s useful for ordinary structs, but counterproductive when a value is meant to remain in a vector or mask register: splitting it can introduce extra moves and obscure what should be a single whole-value store, particularly after inlining introduces more local stores. dotnet/runtime#128013 consistently marks SIMD and mask stores as intrinsic-related across platforms, including 32-bit x86 and x64 mask stores, so those locals remain intact. dotnet/runtime#129563 extends that principle to user-defined structs that are bitcast to SIMD types. This trades away struct promotion for those locals, but enables the JIT to preserve their vector representation.”

Compiler cost models

“A compiler’s cost model assigns estimates to operations and instructions, such as their execution cost or throughput and their impact on code size, and uses those estimates to choose between otherwise legal transformations or instruction sequences. Wrong estimates can still produce semantically correct code, just slower or larger code. With dotnet/runtime#127048, which updates the JIT’s xarch floating-point and SIMD cost model, the JIT’s cost model reflects modern instruction throughput and encoded size, replacing old x87 assumptions and a flat cost for every intrinsic. That leads to better decisions about common-subexpression elimination and loop unrolling, particularly for 512-bit operations.”

Write barriers and the GC

“Write barriers are only one part of the interaction between generated code and the garbage collector. During a compacting collection, the GC needs to plan where surviving objects will move and then update references to them. To do that efficiently, it records their addresses, sorts those addresses, and groups adjacent survivors into regions called “plugs.” With enough live objects, sorting these mark lists becomes a meaningful part of the collection. Recent x86/x64 runtimes use a vectorized vxsort implementation for sufficiently large lists. In .NET 11, dotnet/runtime#110692 from @a74nh extends that support to Arm64.”

CoreCLR runtime replaces Mono on mobile

“Starting with .NET 11, .NET MAUI moved to CoreCLR for Android, iOS, and Mac Catalyst, the last .NET MAUI platforms that had still been using Mono. This is much more than swapping one execution engine for another. Those apps now use the same runtime as ASP.NET Core, cloud services, and desktop .NET, with the same JIT, garbage collector, diagnostics infrastructure, performance improvements, and bug fixes. It also brings CoreCLR’s tiered compilation, ReadyToRun, and profile-guided optimization to mobile, while providing a common foundation for NativeAOT.”

Unnecessary fences on ARM

“[…] removing unnecessary volatile annotations from a wide range of library fields whose correctness already comes from locks, Interlocked, or one-time initialization. On x86/x64 hardware, which already provides a strong memory model, those annotations generally don’t result in extra instructions, though they can still constrain compiler optimizations. Arm, however, permits more reordering, so the JIT often needs to emit memory fences to provide volatile‘s guarantees. Removing the annotations where they’re redundant therefore can end up removing unnecessary fences from Arm’s generated code.”

BigInt rewrite

“[…] rewrote significant portions of BigInteger‘s implementation, changing its limbs (the fixed-size pieces stored in its backing array) from uint to nuint (UIntPtr). That makes no effective difference on a 32-bit machine. On a 64-bit machine, however, each limb grows from 32 to 64 bits; since most arithmetic on a 64-bit value on a 64-bit platform costs no more than the corresponding 32-bit operation, each step can therefore process twice as many bits in the same number of cycles.”

This is only the beginning of the improvements for BigInts. The improvements were applied for larger floating-point formats, like Decimals as well. Basically, all of the big number-crunching involved in processing models, ML, cryptography, etc. are all significantly and noticeably faster and more memory-efficient in .NET 11.

Miscellaneous

  • Ditto for UT8 support: it’s handled and compiled much more efficiently than previous .NET versions, with the low-level performance improvements showing up everywhere.
  • Ditto for RegEx, which, while previously having gotten a lot of love with source-generation and improvements to the engine, still had performance-improvement potential in .NET 11. For example, the results compiled RegExes have been brought in line with the previously better-optimized source-generated C# RegEx code.
  • Many of the improvements in searching are to improve mapping to instruction primitives on the ARM architecture, especially ARM64.
  • There were some relatively minor Linq and IO improvements, as well as some more impactful networking improvements, especially on Linux, where the mapping to platform APIs and primitives has been improved.
  • JSON-writing was significantly improved in some cases.