The article jj init — Sympolymathesy by Chris Krycho explains what Jujutsu is and what it does. I was reminded of these notes that I wrote over a year ago when I read Evolving Git for the next decade by Joe Brockmeier (LWM.Net), which briefly mentioned it as a command-line UX toward which Git itself is working. [1]

Git is not worse than all the others

“Jujutsu is two things: It is a new front-end to Git. This is by far the less interesting of the two things, but in practice it is a substantial part of the experience of using the tool today. In this regard, it sits in the same notional space as something like gitoxide. Jujutsu’s jj is far more usable for day to day work than gitoxide’s gix and ein so far, though, and it also has very different aims. That takes us to: It is a new design for distributed version control. This is by far the more interesting part. In particular, Jujutsu brings to the table a few key concepts — none of which are themselves novel, but the combination of which is really nice to use in practice: Changes are distinct from revisions: an idea borrowed from Mercurial, but quite different from Git’s model. Conflicts are first-class items: an idea borrowed from Pijul and Darcs. The user interface is not only reasonable but actually really good: an idea borrowed from… literally every VCS other than Git.”

I don’t think that this is true. but I have perhaps more experience with “literally every VCS” than the author does. For example, Perforce changelists are anything but intuitive for new users. TFS and Subversion branches were a horror to deal with. Every feature of VSS was only tolerated because it was amazing to have source control at all.. It’s not just Git that has difficulties with UX.

The author is being a dick here, taking easy swipes that they know no-one will question. They just expect to get an amen from their chorus who think that, because there is room for improvement in the Git UX, that every other tool must be better. This is not true. But it doesn’t matter because Git can’t just be an amazing tool for which GUIs have filled the gaps in its UX: it has to suck in order for Jujutsu to save us all from it. People are tiring.

It’s a Google thing

“[…] given it is being actively developed at and by Google for use as a replacement for its current custom VCS setup, it seems like it has a good future ahead of it.”

That is such a naive thing to say, given the history of most of the software that comes out of Google. The best that you can hope for is that Google doesn’t try actively kill it while an open-source community tries to keep it alive.

Jujutsu’s features

“Jujutsu has two discrete operations: describe and new. jj describe lets you provide a descriptive message for any change. jj new starts a new change. You can think of git commit –message “something I did” as being equivalent to jj describe –message “some I did” && jj new. This falls out of the fact that jj describe and jj new are orthogonal, and much more capable than git commit as a result.”

“[…] a given change logically the child of four other changes, with identifiers a, b, c, and d? jj new a b c d. That’s it. One neat consequence that falls out of this: a merge in Jujutsu is just jj new with the requirement that it have at least two parents. (“At least two parents” because having multiple parents for a merge is not a special case as with Git’s “octopus” merges.)”

When would you need this? Am I missing a use case? I’ve never felt particularly like I needed to merge four branches together but I’m just a simple guy with simple needs. I’m sure I’m missing something by not wrangling four simultaneous branches instead of programming. Is this something that LLMs need maybe?

“[…] you can describe the change you are working on and then keep working on it . The act of describing the change is distinct from the act of “committing” and thus starting a new change. This falls out naturally from the fact that the working copy state is something you can operate on directly: akin to Git’s index, but without its many pitfalls.”

What pitfalls does Git’s index have? It seems pretty useful to me. At the end of the article, he actually backs off on this and admits that Git’s index is kinda necessary for staging parts of the workspace. Also, the whole feature—separating “describing” and “committing”—isn’t as revelatory as he’s making it out to be.

“With jj new -A <some change ID>, you just insert the change directly into the history. Jujutsu will rebase every child in the history, including any merges if necessary; it “just works”. That does not guarantee you will not have conflicts, of course,”

Um. Ok. I mean, how could it possibly have guaranteed that? It’s not magic. Conflicts are the nasty part, though, and are always more difficult to deal with when doing operations on multiple commits at once.

Delaying conflict-resolution is cool

“Jujutsu can incorporate both the merge and its resolution (whether manual or automatic) directly into commit history. Just having the conflicts in history does not seem that weird. “Okay, you committed the text conflict markers from git, neat.” But: having the conflict and its resolution in history, especially when Jujutsu figured out how to do that resolution for you, as part of a rebase operation? That is just plain wild .”

I kind of like the idea of committing conflicts instead of forcing the user to resolve them immediately. That’s a good idea. It lets you revisit the merge by preserving the inputs.

“Jujutsu will add conflict markers to a file, not unlike those Git adds in merge conflicts. However, unlike Git, those are not just markers in a file. They are part of a system which understands what conflicts are semantically, and therefore also what resolving a conflict is semantically. This not only produces nice automatic outcomes like the one I described with my library above; it also means that you have more options for how to accomplish a resolution, and for how to treat a conflict. Git trains you to see a conflict between two branches as a problem. It requires you to solve that problem before moving on. Jujutsu allows you to treat a conflict as a problem which [must eventually] be resolved,”

“Jujutsu allows you to create a merge, leave the conflict in place, and then introduce a resolution in the next commit, telling the whole story with your change history.”

“Conflicts are inevitable when you have enough people working on a repository. Honestly: conflicts happen when I am working alone in a repository, as suggested by my anecdote above. Having this ability to keep working with the repository even in a conflicted state, as well as to resolve the conflicts in a more interactive and iterative way is something I now find difficult to live without.”

The article discusses retaining conflicts in commits as first-class, semantically valuable artifacts that the conflict resolver can either resolve immediately or later when another commit comes along to make the conflict go away. This is very interesting for multi-commit rebases where git currently makes you resolve the conflicts every step of the way, even when you know that the conflict will definitely go away further up the chain.

Often, you don’t even remember how you actually want to resolve the conflict in the “old” commits—and you don’t care. This only happens with rebase, which I use much, much, much more than merge. When you merge, git considers the sum of all changes in all commits that you’re merging, so you get the behavior you want: the sum of the commits eliminate irrelevant conflicts. Rebase in git doesn’t benefit from this behavior. In jujutsu, it does.

Massaging your commits is good, too

“[…] this ability to move part of one change into a different change is a really useful thing to be able to do in general. I find it particularly handy when building up a set of changes where I want each one to be coherent — say, for the sake of having a commit history which is easy for others to review.”

Yes, of course. Some of us have just been using powerful GUIs to do this for a long time, with hotkeys flying, instead of furiously typing commands into a command line. This is a laudable goal. I’m glad that jujutsu brings that experience—and that ability—to command-line users.

Wait. No branch names?

“The default log template shows me the current set of branches, and their commit messages are usually sufficiently informative that I do not need anything else.”

Sure, ok. No branch label required. That’s definitely not a recipe for disaster for most developers. It never struck me as too burdensome to have to make little bookmarks, really. It felt kind of obvious and logical. I guess I didn’t notice that I was wasting time when I could have just been reading commit messages instead.

Acknowledging Git GUIs

“GUI tools could make all of those much easier. Any number of the Git GUI s have tried, but Git’s underlying model simply makes it clunky. That does not have to be the case with Jujutsu. Likewise, surfacing things like Jujutsu’s operation and change evolution logs should be much easier than surfacing the Git reflog, and provide easier ways to recover lost work or simply to change one’s mind.”

First of all, I don’t think there’s that strong a point to make here. A GUI like SmartGit manages to elide a lot of the complexity and has done so for years.

I wonder if Syntevo is working on anything for jj? Or is there no point because SmartGit has actually already been doing what Jujutsu does for a while? Maybe there is an advantage to storing the conflicts that SmartGit could also leverage.

Editing history for code you don’t have checked out

The article I’m daily driving Jujutsu, and maybe you should too by Drew Devault writes,

“As a git power user, I rely heavily on git rebase to edit my git history as I work, frequently squashing and splitting and editing commits as I work, and I used “stacked diffs” without branches before it was cool.”

Same. I’ve just never done it on the command-line, so I’ve never felt the pain of doing any of this git-fu there. SmartGit makes most of my history-editing seamless, easy, and foolproof. I know all of you console-jockeys hate them but give me drag-and-drop operations any day.

“When I edited this earlier commit, I was in the middle of working on something else and I hadn’t committed or even staged it. I did not run git stash, nor git commit -m”WIP”, nor git add, nor git checkout, nor git rebase, at any point. The only command I ran was jj squash. When it was done, I was returned immediately to where I left off, with a half-written, uncommitted change in my workdir. It took all of two seconds to complete this operation and pick up where I left off.

“The “wow” moment came when I realized that I had done this several times that day without finding it particularly remarkable. Jujutsu makes editing history absolutely effortless.”

This is, I think, the killer feature of jujutsu: you can edit history that you don’t currently have checked out.

While better conflict-handling and editing commits anywhere in the tree are attractive features, I still feel that switching back to using the command-line would incur enough drawbacks to outweigh the benefits for me.

Jujutsu Kaisen is a completely unrelated anime

In the video Context is Everything by Andreas Fredriksson (Vimeo), the author pinpoints that a dependency in his app—a JSON-handling library—is sucking all the performance out of it.

So, he takes a look at it.

It’s a general-purpose library, with a lot of edge cases…edge cases that his input data doesn’t have. That is, if he can guarantee a certain context, then he can use an optimized version of the JSON library’s code. This isn’t always going to be the solution—it will, in fact, rarely be the solution for a LOB app for which every line of maintenance is a burden—but, when you’re making something with performance constraints, it’s good to be able to think like this. [1]

He takes the original JSON library and profiles it. Then he starts to excise the slow bits—bits his app doesn’t need anyway. This gets him impressive performance boosts.

First, he gets it to be 2x faster with a simple linear fix (removing unneeded branches), then boosts it to over 11x faster by using a mixed-parsing mode.

Another profile shows that a function called isspace() is taking up 45% of the processing time now. He trims that down to just handle the whitespace characters his file might actually contain. He also ditches the locale check that happened every single time.

17x faster now.

OK. What else can we do? Ah, we could observe that the data doesn’t have to contain spaces at all! That is, instead of parsing the spaces as they come along, you can use a SIMD-based solution combined with a LUT (Look-Up Table) to normalize the input data before you even parse it. He uses a quick-and-dirty Perl script to build the LUT.

22x faster now.

That performance improvement alone is 5x more than the original speed of the parser.

• We just removed a bunch of poorly predicted branches, nothing else
• Low-level thinking = not paying for things you don’t need
• Low-level thinking = partition work in hardware-friendly ways

“[…]

“We didn’t change any of the behavior of the program. All we did was we separated these two passes in a way that was friendly for the hardware. We moved branches from being in the integer control flow to being inside masks in the SIMD flow.”

The next step is to reexamine what “white space” actually is: he reinterprets it to mean anything that’s not a printable character, which allows him to optimize the mask even further.

29x faster.

Over 1GB/s of throughput.

Are we done? Bitch, please.

He moves on to two more levels of optimization that still bring good-sized gains, but at the cost of more complexity. They also contain more assumptions but that’s OK if the assumptions will always be correct. You want to stop optimizing when it makes sense for your use case. If you’re writing code for a very tight loop on some low-level hardware—or in a game where the budget per frame is a maximum of 16ms—then it might be very important: you might end up saving incredible amounts of time for your users; you might be using a lot less power.

• Solve the right problem

  • Ask the right questions
  • Consider the liabilities and overall economics of your approach
• Consider the unique context and the potentially massive wins

  • Generic means “not tuned for your use case”
• Don’t be afraid to look inside

OCaml LogoI don’t program with OCaml. I never have. I have a good colleague who does, occasionally, write stuff in OCaml, and he sent me a bunch of links about OCaml Effects, starting with a discussion asking Are we rational? About exceptions and effects by olleharstedt (OCaml Community).

The author writes,

“I was thinking about the fact that there’s no consensus about exceptions and whether to include them or not in a programming language. Think about Go. They decided to not add support for exceptions. Did they cite any study to support this decision, that supports the notion that exceptions in general lower the quality[1] of the ecosystem? Not that I know of. Now OCaml goes in the opposite direction − adding more ways to jump around in the code, with effects. Also no studies, no experiments.”

This is an excellent point to make: we consider ourselves to be rational engineers and scientists but, very often, we nearly completely elide our reasoning for major decisions about architecture and functionality. [1]

From there, I was intrigued to learn more about this controversial feature in OCaml and I landed on the documentation page called Chapter 12 Language extensions − 24 Effect handlers (OCaml Manual), which summarizes its topic as follows,

“Effect handlers are a mechanism for modular programming with user-defined effects. Effect handlers allow the programmers to describe computations that perform effectful operations, whose meaning is described by handlers that enclose the computations. Effect handlers are a generalization of exception handlers and enable non-local control-flow mechanisms such as resumable exceptions, lightweight threads, coroutines, generators and asynchronous I/O to be composably expressed.”

While I—someone who’s been reading programming-language specifications for over 30 years—really like the sound of that, I’m forced to admit that most of the documentation, while comprehensible to someone versed in language constructs and terminology, serves as a perfect example of “why no-one uses OCaml.”

It is dense.

Consider the following description of a concept like exception-handling, which many programmers can grok relatively easily, when it’s been abstracted away into a generalized effect mechanism, described as follows,

“We run the computation comp1 () under an effect handler that handles the Xchg effect with a continuation bound to k. Here effect is a keyword which signifies that the Xchg n pattern matches effects and not exceptions. As mentioned earlier, effect handlers are a generalization of exception handlers. Similar to exception handlers, when the computation performs the Xchg effect, the control jumps to the corresponding handler, and unhandled effects are forwarded to the outer handler. However, unlike exception handlers, the handler is also provided with the delimited continuation k, which represents the suspended computation between the point of perform and this handler.”

This is not to take anything away from the documentation—which is precise, indubitably accurate, extensive, and replete with examples—but you’re not going to be onboarding any newcomers or dilettantes with this kind of stuff.

As a sign that even OCaml community members are aware that this might be a problem, the page Concurrent Programming with Effect Handlers (GitHub) offers another view on it. It describes OCaml effects as,

“An algebraic effect handler is a programming abstraction for manipulating control-flow in a first-class fashion. They generalise common abstractions such as exceptions, generators, asynchronous I/O, or concurrency, as well as other seemingly esoteric programming abstractions such as transactional memory and probabilistic computations.

“Operationally, effect handlers offer a form of first-class, restartable exception mechanism. In this tutorial, we shall introduce gently algebraic effect and handlers with gentle examples and then continue on to more involved examples.”

Don’t get me wrong, I find reading about a generalized mechanism that collects all of the effect-ful mechanisms hard-coded into other languages fascinating. With effects, OCaml has sacrificed “elegance and approachability of the language” for “provability of the program”. Even if the number of people who end up using this feature in OCaml amounts to a rounding error, I think that research into mechanisms like this is vital because it leads to improvements in other, more mainstream languages.

Related to this all is a practical implementation using effects for a laudable goal: inversion of control and dependency injection [2], for which the approaches in OCaml are described in detail in Basic dependency injection with objects (Grim's web corner). It discusses two common approaches to DI in OCaml and then proposes a more practical alternative.

On the effect-based approach, the author writes,

“an Effect system is often described as a systematic way to separate the denotational description of a program, where propagated effects are operational “holes” that are given meaning via a handler, usually providing the ability to control the program’s execution flow (its continuation), unlocking the possibility to describe, for example, concurrent programs.”

“It’s quite amusing to see that dependency injection and exception capturing can be considered two special cases of effect abstraction, differing only in how the continuation is handled.”

Spoiler: the author ends up using objects rather than modules (weak type-inference support, overly verbose) or effects (weak type-inference support, complexity).

So it all started with the following line of code in the Startup.cs of a WPF application,

locator.GetInstance<IAuthenticationService>().LogInBasedOnGeneralSettings();

It was to be replaced with these lines of code.

#if DEBUG
    locator.GetInstance<IAuthenticationService>().LogInBasedOnGeneralSettings();
#else 
    locator.GetInstance<LoginViewModel>().Show();
#endif

Reduce startup complexity

Going by the single-responsibility principle, the startup should be responsible for starting the app but not for making decisions.

The new code makes a decision, so it should be encapsulated in a component.

The simplest (though not testable) way of reducing cognitive complexity (SonarCube) (PDF Download) is to move the logic to a function. E.g.,

private void EnsureLoggedIn(IServiceLocator locator)
{
#if DEBUG
    locator.GetInstance<IAuthenticationService>().LogInBasedOnGeneralSettings();
#else 
    locator.GetInstance<LoginViewModel>().Show();
#endif
}

Now the calling code is much simpler,

 EnsureLoggedIn(locator);

Define a login service

I also like to reduce calls to locator.GetInstance() as much as possible, so I prefer do define something like a LoginService that consumes the IAuthenticationService and the LoginViewModel, so that you have something like this:

class LoginService(
    IAuthenticationService authenticationService,
    LoginViewModel loginViewModel)
{
    public void EnsureLoggedIn()
    {
#if DEBUG
        authenticationService.LogInBasedOnGeneralSettings();
#else 
        loginViewModel.Show();
#endif
    }
}

Now we can make the method in the startup file use the LoginService instead.

locator.GetInstance<LoginService>().EnsureLoggedIn()

We don’t need to pollute the startup with the nuance of which mode we’re in.

Making it testable

A colleague responded that,

“But then you have to […] inject a ViewModel into a Service?”

I’ll include the reply to show how the sausage is made: instead of just showing the solution, I often appreciate learning how people think about problems.

So, here was my response,

I’m not trying to be pedantic; it just comes naturally. 😃

• I was going to write that injecting a ViewModel into a service isn’t bad because it’s just a view model. But then I noticed that it seems to be communicating with the view in order to show something to the user. 😃
• We’re trying to abstract away complexity and to make our logic testable.
• We need to call Show() during startup; that’s a fact. If we introduce a service, it actually makes that part mockable.
• If we wanted to test that the LogInBasedOnGeneratedSettings() is called when expected, we couldn’t do that right now, could we?
• If we make it a service, then we could think about verifying the logic with a test.
• Of course, once we want to build the test, we’d then be confronted with the need to abstract away the compiler-define. Otherwise, we wouldn’t be able to test both branches without recompiling. That’s a code smell, too.
• Which is why I usually end up with some standard settings objects like the following.

public interface ICompilerSettings
{
    public bool IsDebug { get; }
}

public class CompilerSettings : ICompilerSettings
{
    public bool IsDebug
    {
        get
        {
#if DEBUG
        return true;
#else 
        return false;
#endif
        }
    }
}

public interface ILoginServiceSettings
{
    public bool ForceLogin { get; }
}

public class LoginServiceSettings(ICompilerSettings compilerSettings) : ILoginServiceSettings
{
    public bool ForceLogin => !compilerSettings.IsDebug;
}

You might think that this is over-engineering, overkill, an excess of ceremony introduced by an architect astronaut!

I, on the other hand, think that this is a minimal solution that separates concerns and makes all branches testable.

Once we inject the ILoginServiceSettings into the LoginService, we can easily verify the behavior with tests (using fakes, mocks, etc.).

class LoginService(ILoginServiceSettings settings,
    IAuthenticationService authenticationService,
    LoginViewModel loginViewModel)
{
    public void EnsureLoggedIn()
    {
        if (!settings.ForceLogin())
        {
            authenticationService.LogInBasedOnGeneralSettings();
        }
        else
        {
            loginViewModel.Show();
        }
}

It might look like a lot of ceremony but, without it, how else can you say with confidence that the login is required in some cases but not others? We can even verify that it’s not required in DEBUG mode by mocking ICompilerSettings.

Then the only thing we have to verify without automated tests is that the CompilerSettings are implemented as expected, which is very little code to manually check. We don’t need to look at the rest. 👍

We want to test everything

My colleague very politely responded,

“Injecting ViewModels into Services is generally considered bad practice. The rest seems to depend on what you wish to test and don’t overengineer it…”

At this point, we took the conversation to meatspace, i.e., I ran over to his desk to tell him that “I always want to test everything.” I am willing to concede on time constraints, priority, and planning, but my goal is “test all the code paths” eventually. I’m patient, though, so will accept unwritten tests as technical debt. I will design my could so that it could be tested, though, when we eventually find time to do so. This

We shouldn’t just punt on tests because “it looks difficult” or “it’s not much logic”.

In the first case, the fact that it looks difficult may indicate you’re not writing your code in a testable way or may reveal architectural problems. In the second case, those are famous last words. If it’s just a little logic, then why wouldn’t you just test it instead of investing the time arguing that you don’t need a test?

If you have a code base that’s difficult to test because of some unfortunate architectural decisions, then the thing to do is not to ignore it but to slowly chip away at it.

How else would we get a higher percentage of our code covered by tests? Hint: it’s not by continuing to write more code without tests.

Addendum: A note on architectural boundaries

He’d also argued about mixing levels—injecting a ViewModel into a service—but I convinced him that this is already what was happening whether you wrap a service around it or not. The startup is already instantiating and using a view model. Is that somehow better?

I don’t think it’s a bad thing, as it’s just a way of asking the user for input in order to continue starting the application. It’s a step in the application startup. If you wrap it in a service, then you can at least test that the code does what you want. This is exactly the kind of thing that everyone is going to forget to test manually.

Should the startup be using the view? Maybe, maybe not. It currently does and it makes for a legible workflow. Being too pedantic about architectural boundaries just for the sake of it is often wasted effort and can often lead to overly complex solutions.

Ordinarily, the problem would be that introducing a dependency across a boundary weakens testability, but that’s obviously not the case here. We can just mock away the view and test all of our logic.

I wouldn’t worry about this one too much, though it’s a good rule of thumb.

The article C# 14 Extension Members: Complete Guide to Properties, Operators, and Static Extensions by Laurent Kempe writes,

“Perhaps the most powerful C# 14 capability is extension operators. You can now add user-defined operators to types you don’t control, enabling natural mathematical operations.”

When I first saw this, I thought it was kind of gimmick-y. But I just realized why it’s very nice that you can declare operators separately—optionally—from the type. Adding operators by default is a heavy decision in most APIs. You generally don’t do it except for the most obvious cases, like matrices, etc. where there is really only one possible way to implement the standard operators.

However, for a lot of other types, it would be convenient to have these operators but they might be annoying for some. This way, you can either add them in yourself—tailoring the implementation for your needs—or you can pull in a NuGet package that extend standard types with operators. This allows you to opt in to the operators.

With these new extensions, we’re probably going to see more lightweight types that are delivered in multiple NuGet packages, the satellite packages being extensions that enhance the base type for specific scenarios.

The author demonstrates such a custom operator, using tuples.

extension(Point point)
{
    public static Point operator +(Point point, (int dx, int dy) offset) =>
        new Point(point.X + offset.dx, point.Y + offset.dy);
}

// Usage:
Point translated = myPoint + (5, -3);

Nice.

I had never thought of an if statement as a type-check until a Smalltalk programmer explained it to me in this video. She explained how Smalltalk has six keywords—according to Wikipedia, they’re true, false, nil, self, and super, but her list had thisContext on it as well [1]—and you can get rid of conditions and turn them into message-passing instead, as God intended.

RailsConf 2015 − Nothing is Something by Sandi Metz on May 1, 2015 (YouTube)

From the official video description,

“Our code is full of hidden assumptions, things that seem like nothing, secrets that we did not name and thus cannot see. These secrets represent missing concepts and this talk shows you how to expose those concepts with code that is easy to understand, change and extend. Being explicit about hidden ideas makes your code simpler, your apps clearer and your life better. Even very small ideas matter. Everything, even nothing, is something.”

The null-object pattern

Null-object patternI often use sentinel (or placeholder) objects so that I don’t have to query a condition, like if (a == null) { }. Instead, as Sandi says, you just “pass a message”. She calls it the “null-object pattern” or an “active nothing”. Fine, cool. Lots of names for it.

As she also noted, you don’t get rid of the conditional, but you move it to the place where the decision should be made, rather than propagating the decision to every caller or dependency.

She spent a lot of time on it, but it’s basically about the following pattern, which is drastically simplified from what you’d probably find in the wild.

interface  IAnimal
{
    public string Name { get; }
}

class Animal : IAnimal
{
    public string Name { get; init; }
}

List<IAnimal> animals = [new Animal { Name = "Pig" }, null, new Animal { Name = "Cow" }];

foreach (var animal in animals)
{
    if (animal != null)
    {
        Console.WriteLine(animal.Name);
    }
    else
    {
        Console.WriteLine("no animal");
    }
}

The condition is the problem, because every client of that list has to deal with the possibility of nulls. One way to handle it would be to move the condition out of the (obvious) loop, eliding nulls from the list, as shown below.

var actualAnimals = animals.Where(a => a != null);

foreach (var animal in actualAnimals)
{
    Console.WriteLine(animal.Name);
}

You still have the conditional, of course, but you’re also handling it just once and then letting the rest of your code be free of needing to deal with possible nulls.

However, this hides the length of the original list, which is not always what you want. What if you want to represent the “empty” slots? What if, as the talk is called, “Nothing is Something”? Then you would use the “null-object pattern”.

class MissingAnimal : IAnimal
{
    public Name => "no animal";
}

var actualAnimals = animals.Select(a => a ?? new MissingAnimal());

foreach (var animal in actualAnimals)
{
    Console.WriteLine(animal.Name);
}

Voila.

Composition beats inheritance

In the second act of this 36-minute talk, she demonstrates how to use composition rather than inheritance by ruthlessly applying the single-responsibility principle. She starts with a simple-looking class that returns some data.

class Thing
{
    private IEnumerable<string> _data;

    public Thing(IEnumerable<string> data)
    {
        _data = data ?? throw new ArgumentNullException(nameof(data));
    }

    public IEnumerable<string> Data => _data;
}

She then shows how you can use inheritance to make two descendants, one of which returns the data in a random order and another that returns the data with each entry doubled.

class Thing
{
    private IEnumerable<string> _data;

    public Thing(IEnumerable<string> data)
    {
        _data = data ?? throw new ArgumentNullException(nameof(data));
    }

    public virtual IEnumerable<string> Data => _data;
}

class RandomThing
{
    public override IEnumerable<string> Data => base.Data.Shuffle();
}

class DoubleThing
{
    public override IEnumerable<string> Data => base.Data.Zip(data, (x, y) => new[] { x, y }).SelectMany(o => o).
}

Now try to make one that returns the data in a random order and doubles each entry. Don’t repeat yourself.

With inheritance, you’re quickly in a tight spot.

The thing to remember is that you’ve now introduced two new features to Things, which kind of slipped in there: RandomThing orders the data but does not transform it, whereas DoubleThing transforms the data but doesn’t touch the order.

It sounds like the Thing now has two responsibilities, i.e., it addresses two concerns.

The answer is to separate out these two concerns into components and then to inject those components into the Thing. It’s always the same answer. It’s boring, right? Boring is good.

This is an intermediate step, to illustrate the simplest form of composition, with the fewest changes. It’s going to be more code than we’d like, but let’s go ahead and write it.

class Transformer
{
    public virtual IEnumerable<string> Transform(IEnumerable<string> data) => data;
}

class Doubler : Transformer
{
    public override IEnumerable<string> Transform(IEnumerable<string> data)
    {
        return data.Zip(data, (x, y) => new[] { x, y }).SelectMany(o => o);
    }
}

class Sorter
{
    public virtual IEnumerable<string> Sort(IEnumerable<string> data) => data;
}

class Shuffler : Sorter
{
    public override IEnumerable<string> Sort(IEnumerable<string> data) => data.Shuffle();
}

class Thing(IEnumerable<string> data, Transformer transformer, Sorter sorter)
{
    public IEnumerable<string> Data => sorter.Sort(transformer.Transform(data));
}

new Thing(["A", "B", "C"], new Doubler(), new Shuffler());

This is immediately obviously suboptimal First of all, we should recognize that changing the order and transforming the data aren’t different operations. They’re both functions on a sequence that return another sequence. Instead of passing in a Sorter and a Transformer, as in the example in the video, we could instead pass in a sequence of transfomers to apply.

class Transformer
{
    public virtual IEnumerable<string> Transform(IEnumerable<string> data) => data;
}

class Doubler : Transformer
{
    public override IEnumerable<string> Transform(IEnumerable<string> data)
    {
        return data.Zip(data, (x, y) => new[] { x, y }).SelectMany(o => o);
    }
}

class Shuffler : Transformer
{
    public override IEnumerable<string> Transform(IEnumerable<string> data) => data.Shuffle();
}

class Thing(IEnumerable<string> data, IEnumerable<Transformer> transformers)
{
    public IEnumerable<string> Data => transformers.Aggregate(data, (current, t) => t.Transform(current));
}

new Thing(["A", "B", "C"], [new Doubler(), new Shuffler()]);

Another thing we can notice is how rigid this all is in the type of the item. Let’s make this a more generalized pattern.

class Transformer<T>
{
    public virtual IEnumerable<T> Transform(IEnumerable<T> data) => data;
}

class Doubler<T> : Transformer<T>
{
    public override IEnumerable<T> Transform(IEnumerable<T> data)
    {
        return data.Zip(data, (x, y) => new[] { x, y }).SelectMany(o => o);
    }
}

class Shuffler<T> : Transformer<T>
{
    public override IEnumerable<T> Transform(IEnumerable<T> data) => data.Shuffle();
}

class Thing<T>(IEnumerable<T> data, IEnumerable<Transformer<T>> transformers)
{
    public IEnumerable<T> Data => transformers.Aggregate(data, (current, t) => t.Transform(current));
}

new Thing<string>(["A", "B", "C"], [new Doubler<string>(), new Shuffler<string>()]);

Note that now we have all of our logic independent of the type of item in the sequences. It’s only in creating the Thing that you decide on the item type.

The Transformer is called a functional interface—i.e., an interface with a single function—which would be type-compatible with a function signature in Java, but still isn’t in C#. It’s kind of clunky and repeats a bunch of code. Can we get rid of it? Can we also get rid of the dynamic dispatch (i.e., the virtual and override)?

Sure, we can.

class Thing<T>(IEnumerable<T> data, IEnumerable<Func<IEnumerable<T>, IEnumerable<T>>> transformers)
{
    public IEnumerable<T> Data => transformers.Aggregate(data, (current, t) => t(current));
}

new Thing<string>(["A", "B", "C"], [data => data.Zip(data, (x, y) => new[] { x, y }).SelectMany(o => o), Shuffle]);

Well, that’s a lot less code, but it’s a bit messy at the declaration point. One nice thing, though, is that we’re only declaring the item type once now, in the type parameter to Thing.

We can clean that up a bit but we’re going to be limited by the requirement to specify the type parameter as soon as we leave the constructor of the Thing. The Shuffle part is succinct enough but the Double part isn’t at all obvious.

How about we encode the types in a static method call?

public static class ThingTools
{
    public static IEnumerable<T> Double<T>(this IEnumerable<T> data)
    {
        return data.Zip(data, (x, y) => new[] { x, y }).SelectMany(o => o);
    }
}

new Thing<string>(["A", "B", "C"], [ThingTools.Double, Shuffle]);

That’s quite a bit better. Now that we already have a helper class, we can keep improving things by making another helper method that allows us to create a Thing by passing in a collection of items without specifying the item type explicitly. Instead, the item type is picked up from the data passed in.

public static class ThingTools
{
    public static IEnumerable<T> Double<T>(this IEnumerable<T> data)
    {
        return data.Zip(data, (x, y) => new[] { x, y }).SelectMany(o => o);
    }

    public static Thing<T> Create<T>(IEnumerable<T> data, IEnumerable<Func<IEnumerable<T>, IEnumerable<T>>> transformers)
    {
        return new Thing<T>(data,transformers);
    }
}

ThingTools.Create(["A", "B", "C"], [ThingTools.Double, data => data.Shuffle()]);

Isn’t that fun?

Did I go too far? You decide what you want to use based on your comfort level with any of the versions that use composition (don’t use the ones that don’t use composition).

I was recently allowed to observe as a team discussed the benefits and drawbacks of using an IOC container. [1]

I was asked not to directly participate because it was a team-building exercise; the team needed to convince itself based on the merits of its own arguments. If those for the technology were unable to articulate their convictions sufficiently, then it wouldn’t help for an outside authority to dictate the answer. I assisted in the background, with clarification and alternate explanations.

Why no tests?

Some team members had a reasonable hesitation to using an IOC. Why reasonable? Because they’d been hurt in the past by non-pragmatic and overly magical solutions.

The main reason that the other team members wanted to use DI and an ICO container was to improve testability. They also appreciated that a side-effect of DI is that it makes it so much easier not only to reason about your system, but to repurpose parts of it.

The disconnect arose because the first group doesn’t write automated tests. Therefore, they never felt the pain of trying to replace an annoyingly impure component deep in the program logic with something else in order to test other components. If that situation doesn’t comes up, then you might not see what the big deal is.

So, part of the confusion was that some of the team still had an at-best antiquated—and at-worst irresponsible and inefficient—approach to engineering because they did all of their testing manually and in an ad-hoc manner.

Another part of the confusion was terminology, where people were arguing against a technology by naming the concept. This was a missed opportunity for finding common ground and then focusing on the details, where they had different preferred approaches.

Clarifying terminology

My colleagues were contrasting DI with what they were calling “static class trees”. I think that expression is quite confusing because what they probably meant was “static object trees,” which I think was meant to mean the untestable evil that is “bottom-up-instantiated object graphs.”

Let’s be clear about which kind of static class trees we think are bad.

Not using DI

This is a static class tree rooted at D that does not inject any dependencies.

class A {}
class B {}
class C {
    A a;
    B b;
    C()
    {
        this.a = new A();
        this.b = new B();
    }
}
class D
{
    A a;
    C c;

    D()
    {
        this.a = new A();
        this.c = new C();
    }
}

D d = new D();

Using some DI

You might complain that this is pathological because we’ve created A twice. OK, fine, let’s pass it in to C.

class A {}
class B {}
class C {
    A a;
    B b;
    C(A a)
    {
        this.a = a;
        this.b = new B();
    }
}
class D
{
    A a;
    C c;

    D()
    {
        this.a = new A();
        this.c = new C(a);
    }
}

D d = new D();

Congratulations! You’ve injected your first dependency.

Now keep going!

Using DI

The following code illustrates a static class tree rooted at D that uses dependency injection for everything. Note that this example uses primary constructors without confusing things too much, shortening the code considerably. All instantiations are under the control of the calling code.

class A {}
class B {}
class C(A a, B b) {}
class D(A a, C c) {}

A a = new A();
D d = new D(a, new C(a, new B()));

How DI and IOC are related

• IOC is the concept. It stands for “inversion of control”, which means that the control over who gets to decide which implementation backs a given interface is no longer with the consumer of the interface but the provider.
• DI is a way of implementing IOC. (Usually rounded up to be equivalent.)
• An IOC Container services requests for instances based on interface-to-implementation mappings.

Looking at the examples above, I think we can all agree that DI is a good thing. That is, “dependency injection” and “inversion of control” as concepts are good things.

What does the IOC container do?

An IOC container generally consists of two parts:

• A mapping of abstractions to implementations.
• A method to resolve implementations from abstractions.

In .NET, these are two completely separate interfaces, so that you can’t register mappings when you should only be using them. You use the IServiceCollection in .NET to register mappings and then use IServiceProvider to request instances.

The service provider locates a requested service and constructs an instance where necessary. It recursively locates any parameters to the constructor of a requested service. Obviously, a directly requested service must be registered with a concrete implementation. But also, every service on which it depends must also be registered with a concrete implementation, recursively until dependencies don’t have dependencies of their own.

Using an IOC container carries the following implications.

• ✅ It reduces fragility when constructors are refactored.
• ⚠️ It can make it unclear which constructors are called.
• ✅ It can be helpful for implementing very generalized factories (where you inject the service provider into, say, a “plugin factory”).

A white paper I wrote six years ago has an extended example (in Swift, of all things): Encodo White Papers: DI, IOC and Containers (2019).

As in the examples above, and in the extended one below, most of the steps in the paper do not use a container. You can do DI without a container—it just gets kind of tedious and wordy. Let the IOC container do the brain-dead stuff for you.

What’s a DAG?

Some folks might refer DAGs, which are directed acyclic graphs (Wikipedia). This is just another way of referring to the graph of objects represented by the composition root, which is the single object you should create in your program’s root method—often called main, but sometimes it’s just the main file of your application—to create your application.

It’s possible to build a DAG without DI. The second and third examples in the sections above also create DAGs. The reason those DAGs are hard to change or extend is because dependencies are created at lower levels rather than injected. We want a DAG … but not like that. We want a DAG created with explicit dependencies, as illustrated by the first example.

For the most part, it’s not really important what you call it, as long as you end up with testable components.

“Refining” your code (separating pure logic from impure)

Any non-trivial application comprises pure and impure parts. The impure parts are the messy bits that communicate with the unpredictable outside world:

• Reading from the command line
• Reading configuration from a file
• Reading values from the environment
• Reading data from files
• Reading from a database
• Reading user input
• Calling network services
• Etc.

We don’t want unpredictability in our testing application, so we’ll push all of the impure stuff as far out to the edges as possible, leaving a nice, fat pile of pure logic that we can reliably and reproducibly test.

The preceding sections have hopefully convinced you that IOC is useful, and that constructor-based DI is a good way of implementing it. We’ve also discussed the advantages of using an IOC container to improve flexibility and reduce code-duplication.

All of these things are going to help achieve our goal of testing as much of the program logic as possible automatically.

The single-responsibility principle

Components should not only be pure, but should have a single responsibility. That’s the “S” in SOLID.

For functional languages, parameters serve as dependencies; for C# and many other languages, another natural “injection point” is a constructor. Where injection by parameter mixes parameter types—that is, which parameters are data and which are tools for the calculation—a constructor is more clearly a point at which to inject tools.

Now that we have a concept—use IOC to define dependencies as abstractions—and a mechanism—use DI via constructor to inject dependencies—we can write components that address a single responsibility. That is, we have a mechanism for ruthlessly separating concerns.

Components will stitch other dependencies together to accomplish their task (i.e., their sole responsibility). If this stitching code becomes too involved, then the act of stitching might be its own task!

When each component does only a single thing, it is easier to test its logic in isolation. It is, in many cases, trivial.

We can’t test yet! (We need abstractions.)

Do we have testable components yet, though? No. Even if we were to use a container with the examples above, we’re not done yet! Our goal is to make our logic testable with automated tests. None of the examples above is testable because, although dependencies are injected, they are concrete dependencies.

These cannot be replaced with other implementations and thus cannot be mocked away in tests. That is, if the component A above accessed an external service available only in the cloud or when connected to hardware, you cannot test D without having access either to the cloud or hardware because you can only ever pass in A.

We want all modules, high and low, to define and depend on abstractions that can be replaced. This is the “L” in SOLID. A component should receive configuration through an interface rather than either creating it itself or accessing a static, global, concrete instance. When all components receive all external dependencies as other, injected components, it’s extremely easy to both reason about the code and to test it in isolation.

So, to be able to write automated tests, our next step will be to inject abstractions rather than concrete implementations. An abstraction defines a narrow interface that makes as few promises as possible while still fulfilling its task.

In C#, we typically use interfaces or abstract classes. Interfaces are way better. Just trust me.

A more concrete example

Instead of continuing with the toy classes defined above, let’s look at how we would make a part of our program logic more testable.

Suppose you have the following code:

class EmailClient 
{
    void Send(Email email) { … }
}

class SubscriptionManager
{
    void Notify()
    {
        var client = new EmailClient();
        foreach (var email in _subscriptions.Select(CreateEmail))
        {
            client.Send(email);
        }
    }

    Email CreateEmail(Subscription subscription) { … }
}

Now, suppose you’d like to test this code. You can’t test it without an email server configured because the EmailClient is hard-coded. If you invert control, though, you can pass that dependency in to the SubscriptionManager. One way to do this is to pass the dependency directly into the method, like this:

class SubscriptionManager
{
    void Notify(EmailClient client)
    {
        foreach (var email in _subscriptions.Select(CreateEmail))
        {
            client.Send(email);
        }
    }

    Email CreateEmail(Subscription subscription) { … }
}

Is this really solving anything, though? No. The callee is still in control of the type because the type of the parameter is a specific class. The caller has no choice but to pass in an EmailClient, which will try to sent mails to an external server over a network.

In order to support IOC, the callee needs to abstract its requirement. In C#, you use an interface.

interface IEmailClient
{
    void Send(Email email);
}

class EmailClient : IEmailClient
{
    public void Send(Email email) { … }
}

class SubscriptionManager
{
    void Notify(IEmailClient client)
    {
        foreach (var email in _subscriptions.Select(CreateEmail))
        {
            client.Send(email);
        }
    }

    Email CreateEmail(Subscription subscription) { … }
}

We’re done. We’ve implemented inversion of control. The caller now controls the concrete type.

We are also using dependeny injection but of a very manual kind: the caller is expected to provide the email-sending mechanism. This can be inconvenient and can muddy otherwise legible code because each and every caller has to have a reference the thing that the SubscriptionManager needs. That is, instead of coupling just the SubscriptionManager to an IEmailClient, we end up coupling any client of the SubscriptionManager as well.

Therefore, a common practice is to inject dependencies like this through the constructor.

class SubscriptionManager
{
    private readonly IEmailClient _client;

    public SubscriptionManager(IEmailClient client)
    {
        _client = client ?? throw new ArgumentNullException(nameof(client));
    }
    
    void Notify()
    {
        foreach (var email in _subscriptions.Select(CreateEmail))
        {
            _client.Send(email);
        }
    }

    Email CreateEmail(Subscription subscription) { … }
}

The code that calls Notify() no longer has to know anything about the dependency, thus better decoupling the SubscriptionManager interface from its consumers. The SubscriptionManager declares its dependencies in the constructor, which makes good use of that language construct. That is, we’re leveraging the language to improve the clarity of our design.

At this point, we can still construct the SubscriptionManager manually, passing in the concrete type for IEmailClient but we can now also consider using an IOC container, as outlined above.

Another example: Injecting configuration

Let’s take a look at how we can lean on the IOC container to build our application’s configuration.

public class AppSettings :
	ITimerSettings,
	IListenerSettings,
	IPathSettings,
	IBroadcastSettings,
	IDashBoardSettings
{
/* … */
}

At first glance, you may think this is over-engineered, but there’s a good reason for it. There is a single object holding all the settings for the app. But each service only needs to know about one part of these settings. That is, it’s convenient on the implementation side to have a single object handling all settings, but each service should only be coupled to the settings that it uses.

We don’t want to increase coupling in the services just because of how the current implementation works. Therefore, we register the single implementation for all of service-settings interfaces in the IOC container, and each service uses its own settings interface.

Getting the ball rolling

For this section, I’m going to be referencing from two older articles I wrote about a framework I used to work on. It’s fun that nothing has really changed in the last decade.

• Encodo’s configuration library for Quino: part III (2015)
• API Design: Running an Application (Part I) (2015)
• API Design: To Generic or not Generic? (Part II) (2015)
• Quino 2: Starting up an application, in detail (2015)
• Mini-applications and utilities with Quino (2016)

So we can see how to construct an application out of components. We see how to stitch them together with very simple rules. We see how we can test those components.

But…now we want to run the application. We want it to do the thing that it does. Do we just create a component root and call … um … Run() on it?

IServiceCollection services = AppTools.CreateServiceCollection();
IServiceProvider provider = services.CreateServiceProvider();
IApplication application = provider.GetRequiredService<IApplication>();

application.Run();

Are we cool?

Kind of. Like, that works just fine. The articles referenced above provide a lot more background on providing exception-handling, standard logging, command-line support, etc. But the code above is what we’re shooting for, for real applications. What’s a real application?

Let’s flesh this out a bit.

• How many applications do we have?
• What kind of application are we running?
• What even is an application?

Any solution is going to have at least two applications: Whatever the real application is, and one or more test runners. An application is any way of executing part or all of your program logic.

“Real” applications have event loops

A real application might be a console, a GUI, or a server application. These applications have one thing in common: they contain one or more event loops that react to external input.

Most applications have an event loop.

• A console application that “watches” for changes in the file system, updating other files base on that. It exits when the user issues a special command like Ctrl + C.
• A web server that “listens” on specific ports, returning responses to requests. It exits when the user issues a special command like Ctrl + C or when the system terminates the service.
• A GUI that “responds” to user input—mouse and keyboard events—and runs until the user issues a special command to exit.

Run-once applications

Some applications don’t have an event loop They run the parameters through their flowcharts one time and then exit automatically.

• A console application or script that processes a single set of command-line parameters, like processing a file and producing a report.
• A test runner that executes part of the logic in your real application.

Application “actions”

The example above doesn’t show any detail about what the IApplication does when it runs.

As the referenced articles show in more detail, “getting the whole ball rolling” in a nontrivial application always involves several “actions” to execute during “startup”. That is, the application is not just a service collection, but also a list of startup actions, a list of shutdown actions. Each action is created by the IOC container, so it can have all of the services it needs injected into it.

The basic loop in Application.Run() is something like this:

foreach (var action in _startupActions)
{
  action.Execute();
}

The actual implementation ended up being more complicated than this, as noted above, to accommodate general error-handling, async startup actions, debugging comfort, and to support re-running the application, e.g., when it showed command-line help, or when it needed to run a schema-migration for a database. See the linked article for more information.

I heard you like IOC containers…

Another common problem with IOC containers is: how can you dynamically configure the IOC? What if you want to use a AmazonS3Provider by default, but allow a command-line parameter or configuration file to enable an FTPProvider instead?

DI and IOC are awesome, so we want to use them everywhere, right?

But we can’t, can we? As soon as you get a service provider, you can no longer modify it. As soon as we request the IApplication service above, it’s all over for modifying service registrations.

I’ve wrestled with this a lot in the past. The most relevant article is linked above: Quino 2: Starting up an application, in detail.

Basically, the answer is to use two IOCs.

Bootstrap IOC

The first IOC is much smaller and contains registrations for services needed to configure the Main IOC (e.g. configuration-loader, command-line-reader, fs-location-resolver, etc.). These registrations are necessarily a small core of services that cannot be changed by configuration (files, command-line parameters, database values, etc.). That keeps things simple.

Main IOC

Includes all registrations from the Bootstrap IOC, plus overrides that came out of the configuration, plus anything else needed for the main app.

As noted above, an appilcation’s startup and shutdown are lists of actions (discussed in Encodo’s configuration library for Quino: part III).

Specifically, there are actions to execute during,

• the bootstrap phase,
• the main phase,
• and shutdown.

So, the application startup kind of looks like this:

• Configure services and actions for the Bootstrap IOC and Main IOC.

  • Any registration in the Bootstrap IOC is made in the main IOC as well.
  • Crucially, singletons in the Main IOC are the same as those in the Bootstrap IOC.
• Seal the Bootstrap IOC (i.e., get the service provider from the service collection).
• Execute application-startup actions using the Bootstrap IOC.

  • The first few actions will be stuff like “read command line”, “read configuration”, etc.
  • These might alter the registrations in the main IOC and might add or modify actions to execute. That’s OK. It’s not “sealed” yet.
  • Any attempt to alter a registration in the bootstrap IOC results in an error.
  • Modifying an action in the list before the app’s current position in that list will have no effect.
  • At some point, the “bootstrap” actions are finished, and an action executes that “seals” the main IOC from modification.
  • Now we’re in the “classic” app startup.
• Run the main actions.
• Run the event loop or application logic (e.g, fixed handling for command-line parameters).
• Run the shutdown actions.

There’s more documentation but it’s no longer available because Encodo has taken down all public documentation … and we never published the source code as open source. I’m working from memory and my existing articles. 🤷

But that’s the general gist of it. There are clean solutions to anything that might come up. For example, if you need a more “heavyweight” service during the bootstrap—like a database, which you also use in the main application, but which you want to keep configurable—consider making an interface like IBootstrapConfigurationDatabase or something like that, which will be its own singleton and not even available in the main application phase.

I watched/listened to 35 videos, each between 20 and 30 minutes long, and each listed below. I’ve grouped them but retained the order in which I watched them. I’ve left the notes mostly as I wrote them, which is kind of stream-of-consciousness, kind of snarky.

Some videos that I didn’t like got a lot of notes, some videos I liked got fewer notes. It might seem like I hated the video from my snarky notes but I still ended up rating some of them as 🆗, which means that I thought that either the video was well-presented even though the material doesn’t have much broad appeal or that I thought the material was important even though the presentation wasn’t that great.

Don’t read too much into it. The notes are just to help me remember what I learned from the videos—and can hopefully give you a bit of an idea of what the video is about and whether you want to spend 20–30 minutes on it. YMMV.

• C#, F#, & .NET

  • 🆗 Nullable Reference Types: It’s Actually About Non-Nullable Reference Types
  • ✅ Performance Improvements in .NET 10
  • 🆗 Community Toolkit Roundup
  • ✅ C# Features you need Habits you want
  • ✅ Smatterings of F#
  • ✅ Rx.NET status and plans
• Migration

  • ⛔ .NET Scores “A Perfect 10”
  • ✅ .NET Framework 4.8 to .NET 9 Step by Step
  • ⛔️ Modernizing .NET Applications for the Cloud
• Tools

  • 🆗 Visual Studio Debugger: Advanced Techniques
  • 🆗 New dotnet test Experience with Microsoft.Testing.Platform
  • 🆗 What’s New in NuGet
  • 🆗 Real-World .NET Profiling with Visual Studio
• UIs

  • ⛔ What’s New in Windows Forms
  • ⛔ Modern Windows Development with .NET
  • ✅ TUIs Are Back (Although They Never Left): Creating Modern CLI Apps in .NET.
  • 🆗 Ship Faster with .NET MAUI: Real-World Pitfalls and How to Nuke Them
  • ✅ Building Rock-Solid Avalonia Apps A Guide to Headless Testing with AI Assistance
  • ✅ What’s New in .NET MAUI
  • ✅ Build better web apps with Blazor in .NET 10
• Aspire

  • ✅ Taking .NET out of .NET Aspire − working with non-.NET applications
  • ⛔ From Architecture to Docs: .NET Aspire Documented with Copilot
  • ✅ Windows 365 Meets Aspire − Supercharging Multi-Repo Microservice Productivity
  • ✅ Deep Dive: Extending and Customizing Aspire
  • ✅ Aspire Unplugged with David and Maddy
• Cloud

  • 🆗 What’s New in Containers for .NET 10
  • ✅ What’s new in Azure App Service for .NET developers
  • 🆗 Carbon Aware Computing − Using .NET Open Source libraries for more sustainable applications
• AI

  • ⛔️ Architecting an AI-Powered Sales Dashboard with .NET MAUI and Azure OpenAI
  • 🆗 One Question, One Answer: Designing Seamless AI Agents with C#
  • ✅ Overcoming the limitations when using AI
• Everything else

  • 🆗 Modernizing a 17th Century Italian-English Dictionary
  • 🆗 Going Passwordless − A Practical Guide to Passkeys in ASP.NET Core
  • ✅ GitHub Actions DevOps Pipelines as Code using C# and Cake SDK
  • 🆗 If .NET brewed beer…

Microsoft Ignite 2025

C#, F#, & .NET

🆗 Nullable Reference Types: It’s Actually About Non-Nullable Reference Types by dotnet | Shawn Wildermuth (YouTube)

This is a decent, thorough—though somewhat slow—introduction to non-nullable reference types in .NET/C# (which have been available since .NET 3.x / C# 8). If you already know about them, then there’s nothing new here.

✅ Performance Improvements in .NET 10 by dotnet | Stephen Toub (YouTube)

An in-depth examination of performance improvements in .NET 10. He explains how the various compilers (AOT, JIT, etc.) have been optimized to eliminate allocations and just generally optimized for performance. A reduction in allocations is a multi-win: the performance is better because the allocator isn’t working, the memory usage has dropped, and the garbage collector also works less.

He compares .NET Framework 4.8 vs. .NET 9 vs. .NET 10. The most impressive improvements are from 4.8 to 9.0, of course, but he highlights some interesting places where .NET 10 eclipses .NET 9, where .NET 9 had already eclipsed .NET Framework 4.8.

The last example shows how regular expressions have been continually optimized so that an operation that took 24ms in .NET Framework 4.8 was improved by about 12x to 2.5ms in .NET 9 but has been further improved by about 62,500x to about 40ns in .NET 10.

For more coverage, see Toub’s 232-page tour-de-force on performance in .NET 10.

🆗 Community Toolkit Roundup by dotnet | Gerald Versluis, SergioPedri, Michael Hawker (YouTube)

They spent some time touting the benefits of the toolkits.

• There is an introduction to improvements to the MVVM toolkit.
• There is also a toolkit for Aspire, which is interesting.
• Then there’s the Maui MVVM toolkit, which adds a bunch of media support.
• The Windows toolkit added a lot of fixes and controls for WinUI3.

They note that a lot of stuff incubates in the toolkits and is often migrated to the official libraries after a while.

✅ C# Features you need Habits you want by dotnet | Bill Wagner (YouTube)

He introduces an existing “magic 8-ball” program, demonstrating its functionality. He doesn’t show any tests, though. That does not stop him from refactoring the app to take advantage of “newer” C# features. I write it in quotes because, while some of the features he shows aren’t necessarily new, it’s good to have a video that shows how you should be upgrading your types when you touch old code, to take advantage of better type-checking, to convert potential runtime errors to compile-time errors.

• non-nullable references.
• required and init properties.
• The field element for properties, which is new to C# 14.
• The System.Threading.Lock type instead of System.Object, which allows the compiler to generate more efficient code, all without any change in behavior of the application.
• Using verbatim strings and the newer multi-line verbatim strings.
• Collection expressions. (He explains how the compiler can optimize the capacity for a collection expression, where it cannot for a direct instantiation of new List<T>().)
• The spread operator. (He uses this to replace the explicit call to ToArray(). Again, it’s easier to read and the compiler has more optimization opportunities.)
• The with keyword. (He explains how this allows you to more easily work with immutable types and structures.)
• Using a readonly struct (This sets immutability, which also allows much better optimization, such as lowering copying/allocation when passing data through function/stack boundaries.)

  He optimizes his pattern-matching, where the compiler helps a lot to figure out exactly how much information is needed in the pattern. If a case can’t be reached, it’s an error. He removes the lower-bound check on several cases because they’re not needed. If you remove too much, the compiler tells you.

  AnswerType type = randomIndex switch
  {
      >= 0 and <= 5 => AnswerType.Affirmation,
      >= 6 and <= 9 => AnswerType.Encouraging,
      >= 10 and <= 13 => AnswerType.Uncertain,
      >= 14 and <= 16 => AnswerType. Doubtful,
      >= 17 and <= 18 => AnswerType.Rejection,
      19 => AnswerType.Redo,
      _ => AnswerType.Uncertain
  };

  The following is equivalent:

  AnswerType type = randomIndex switch
  {
      <= 5 => AnswerType.Affirmation,
      <= 9 => AnswerType.Encouraging,
      <= 13 => AnswerType.Uncertain,
      <= 16 => AnswerType. Doubtful,
      <= 18 => AnswerType.Rejection,
      19 => AnswerType.Redo,
      _ => AnswerType.Uncertain
  };

  If you were to change the order of the cases, putting the <= 13 case at the top, the compiler warns that the <= 5 and <= 9 cases will never be matched.

  AnswerType type = randomIndex switch
  {
      <= 13 => AnswerType.Uncertain,
      <= 5 => AnswerType.Affirmation,  // Compile error.
      <= 9 => AnswerType.Encouraging,  // Compile error.
      <= 16 => AnswerType. Doubtful,
      <= 18 => AnswerType.Rejection,
      19 => AnswerType.Redo,
      _ => AnswerType.Uncertain
  };

✅ Smatterings of F# by dotnet | Matthew Watt (YouTube)

The first five minutes is an introduction to the programmer himself, which was a bit odd but it’s fine. It just might not be very interesting if you’re looking for technical guidance.

He moves on to an introduction to his blog, which he wrote with F# on the back-end, and React for the front-end. The comments section that he built uses Elmish, which is a library for emulating the highly functional Elm pattern of building code. The whole web site is functional from top to bottom so it’s kind of neat to see how that works for a real-world application.

He finishes up with five minutes on contributing to open-source code. Again, a nice touch.

✅ Rx.NET status and plans by dotnet | Ian Griffiths (YouTube)

He discusses some examples of some new methods in the 6.1 release. These are quite nice, and the concept of RX is just neat, even though I’ve only ever played with it rather than used it in production.

He discusses in detail how some of the new handling for exceptions “bridges between RX’s world of observable streams and more ordinary async programming.”

In the next section, he discusses how the RX project had to do some extra work because System.Linq.Async is no longer their responsibility. It’s now in the standard library. But they had to make sure that their version gets deprecated in favor of the new one. As a library developer, think that this detail is fascinating, because you can see the the tools available for managing changing APIs and dependencies have gotten quite good.

Finally, he discusses the feature set for Rx.NET 7.0. The functionality won’t change much; it’s mostly library and platform-compatibility. There is a fix for the “bloat” issue, which only affects projects that target UI applications on Windows. It turns out that design decision in version 4.0 left self-contained deployments with implicit references to UI frameworks, which add dozens of megabytes needlessly.

The fix causes a compile error, for which they added an analyzer that nicely explains the fix to apply. This is a neat example of how to help consumers of your library get around compiler errors, which we didn’t have available before it was so easy to write and include custom analyzers. Previously, you’d have had to jump through more hoops to avoid giving upgraders compiler errors that weren’t warnings in the previous version. Now, if something like that is unavoidable, then you can still provide guidance with a diagnostic.

I thought it was a very interesting presentation but I’m a library and framework geek. Your mileage may vary.

Migration

⛔ .NET Scores “A Perfect 10” by dotnet | Shaun Walker (YouTube)

He describes a successful migration of a large Blazor application to .NET 10 (the open-source Oqtane (GitHub)), presumably from .NET 8. This is OK, but he just describes what he did without showing it. Once he gets to the product, he actually ends up demoing the Oqtane software—and Blazor’s capabilities—more than he showed any details about what migrating to .NET 10 entailed, apart from a few sentences in the slides. Instead, he spent a bunch of time discussing features introduced by .NET 10 that Oqtane ended up using. That is, instead of covering the migration itself, he discussed the extensions to the product that were enabled by a move to .NET 10.

✅ .NET Framework 4.8 to .NET 9 Step by Step by dotnet | Michael Christiansen (YouTube)

He recommends modernizing the app before retargeting it. This means:

• Updating to use the SDK-style project format.
• Using package references.
• Using the Microsoft.Extensions.* packages, like dependency injection, configuration, logging, and hosting, all of which target the .NET Standard API surface and are therefore available for .NET Framework and .NET.

After that, he recommends side-by-side versions of libraries so that you can split them up better without affecting the existing, working version of the code.

One of the projects was a tougher nut to crack: it was an old-school ASP.NET application, where the patterns had completely changed in .NET 9 and 10. For that, he managed to have Claude Code do about 90% of the conversion and finished it up manually. The process was very manual—“spec-driven development” and “very hands-on”—but Claude Code was quite helpful once he figured out how to steer it properly.

If you have a .NET Framework application, then this is a great video. He really has a lot of good advice for how to avoid certain pitfalls (e.g., platform-specific code, like Windows Services).

⛔️ Modernizing .NET Applications for the Cloud by dotnet | Matt Soucoup (YouTube)

Was there ever going to be a chance that he wouldn’t start off with telling you that Copilot can do all of the tedious work for you? No. No, there wasn’t. Was he ever going to tell you to use your mad skillz with your IDE to apply a ton of changes automatically using tools and refactoring? No, he wasn’t. Like the NuGet guy, he’s going to get copilot to spend ten minutes running a NuGet one-liner.

So like how cool is that? Not only do you have a super-old application that you never upgrade but now you don’t even have to understand what you’re migrating to! I love how he says that going from .NET Framework to .NET 10 is just soooo easy. You know, don’t make any stops along the way, just take the express train. What could go wrong?

Anyway … he shows how to install the Copilot modernization tools, then opens a .NET Framework IIS-based project. Once again, we’re watching a guy watch a Copilot chat window write a ton of text that he barely reads. He asks it to explain the security problems, as if this is something that you should do. Shouldn’t you inform yourself about the packages? Shouldn’t you just upgrade the old things? Do you really need the explanation?

And, once again, he says that “you’re giving up the reins to Copilot,” but, like everyone else, just assumes that everything that Copilot returns in bulletproof. This is still not my experience, to this very day.

Back to the update plan: I see the attraction, I really do. It’s very detailed … but who is it for? Is he keeping this upgrade plan in the repository? How much control does the plan actual give him? Doesn’t the commit that results just show the changes?

He says it “took about an hour to upgrade”. 😱 Oh, hell no. It just works for an hour for what he calls “a simple app”, using God knows how many tokens, and then you still have to review everything? Why not just do it yourself? He really needs to show the diffs. Show us the diffs, bro. I don’t think he’s going to show us the diffs. He’s just going to show us how he has to coddle the tool, which is basically making black-box changes. “That’s just the way it is, that’s the way it is working with AI-assisted dev tooling.”

He didn’t show the diffs. I have no idea what this tool did for him on this project. This tool is for people who would have no idea how to go about upgrading a solution on their own, who can use a chat windows but run screaming from a command-line upgrade tool.

At the very end, he runs the upgraded version but there are warnings in the build that two packages were restored using .NETFramework,Version=4.6.1 (the worst .NET Framework version ever), which strongly indicates that, even after an hour of f@&king around, the solution still references .NET Framework.

“The amount of coding that I had to do was basically zero. All I had to do was supervise things.”

Well done, buddy. You still have old packages and weird references. Check your warnings. I wouldn’t touch this tooling with a ten-foot pole.

As I wrote in a comment on the video,

This kind of workflow doesn’t translate well to a nearly half-hour-long video. There’s nothing to see. He ran a command or two. He didn’t even show the diffs at the end, to show us what the tool actually did. You could still see some odd warnings about .NET Framework in the output that he had to pretend weren’t there. He did a good job FWIW but a lot of this video is watching the Copilot chat window scroll by. The explanation is good but it would have been better as a blog post.

Tools

🆗 Visual Studio Debugger: Advanced Techniques by dotnet | Harshada Hole (YouTube)

She takes us through the various live and inline indicators in the debugger, with predictive evaluation, including highlighting of the particular part of a condition that caused it to evaluate to true or false. The debugger has moved much closer to Rider’s, showing a lot of calculated values in the whitespace next to code, so you can see return values and calculated values without having to look in the variables or watches panes. This also allows you to use more concise coding while still being able to see interim values while debugging.

When showing how to analyze exceptions, she showed how to dig down into the call stack to find out why something’s null. She used right-clicking for everything, which was already slower than it needed to be…but then she decided to ask Copilot. The “quick” analysis took 30 seconds and then she had to ask it to do a “deep analysis”, whereupon it found the error that she would have probably found manually much more quickly. Maybe a more complex example wouldn’t have had such an obvious fix. Most people suck at debugging and don’t really understand their code, so probably Copilot is better at this than they are (or ever will be). So who am I to stand in the way of progress? I’m just John Henry.

I cannot stress enough how annoying it is to have to watch people “ask Copilot” and then we all gather around the chat-window output like it’s the word of God. It’s too bad, because the first few minutes of this video showed interesting deterministic tools before devolving into an orgy of just clicking that stupid little Copilot icon everywhere and then watching the completely useless and always-disregarded text in the chat windows scroll by. I cannot recall any one of these presenters actually reading any of this text. No-one cares.

These tools are really trying to reach out to and onboard completely unskilled developers to an unprecedented degree. These kinds of presentations make me sad. It’s fine for what it is, but I don’t think that this is the final form of software-development.

🆗 New dotnet test Experience with Microsoft.Testing.Platform by dotnet | Jakub Jares (YouTube)

This is a demo video, with the presenter working in Visual Studio Code but only from the command line. He shows how the console UI has been considerably improved. He also gets into new analyzers, assertions, and attributes. The improvement to the assertions is that they start analyzing the expression tree, which I find to be more fragile than the NUnit approach, which uses an explicit API to declare the assertion, with no magic. The attributes are for extending the framework, e.g., for determining when and in which environments tests will run.

Finally, he shows how the MSTest runner has massively improved execution speed, not in this version (4.0), but already in the 3.0 version.

The video is OK but the product is quite exciting, as it is a massive improvement over the previous test-runner.

🆗 What’s New in NuGet by dotnet | Sean Iyer & Nikolche Kolev (YouTube)

He starts off by threatening us that he will show a bunch of AI stuff. First up: tell us to use the MCP server for NuGet. He uses it to show how to get Copilot to update your dependencies when you have a vulnerability. This is not a hard task and, honestly, you should be aware enough of your dependencies to solve them yourself. It’s nice that the warnings are so good now that you can get a tool to fix up all f the things that people never could figure out on their own. Dude, since assembly-binding redirects were fixed in .NET, there’s no problem anymore. I don’t understand how it’s secure to let a hallucinating machine pick your dependencies for you. Now you don’t have to understand anything!

He spends a bunch of time talking about how to avoid getting outdated implementations that aren’t in the training data using an MCP. Or you could, you know, just update to the latest version. I don’t know why they’re making everything so complicated.

In the second half, he talks about security improvements but then just starts talking about how Copilot did all of his work for him. So, like, it’s secure but also an only partially reliable machine made all of the changes and he didn’t seem to look at them.

Nikolche shows how to eliminate vulnerabilities without Copilot (thank God) and shows how to use the pruning option with the audit command to remove unneeded dependencies that might show up in audits unnecessarily.

🆗 Real-World .NET Profiling with Visual Studio by dotnet | Nik Karpinsky (YouTube)

The first four minutes is a discussion of what profiling even is, with a nice workflow diagram for noobs. Next, he grabs the NLog open-source repository and opens the solution in Visual Studio.

“Now I want to talk to the profiler agent.”

Oh no.

He has the agent build a benchmark for a given class. The build fails, though because the solution uses advanced trimming options. Of course, he can figure this out, but if a developer who needs an agent to write benchmarks gets this failure, their day is already over. Copilot is not going to figure something like this out, either.

He goes on to generate more code but it’s very clear that the agent is a support tool because he brings a lot of knowhow to the table. For example, he sees immediately that the agent’s proposed solution never cleared the StringBuilder, which would skew the results toward better initial performance because of thrashing caused by reallocation that affects only subsequent runs. Of course, if you don’t notice this, then you have a shit benchmark that you will trust unreservedly because we’ve all long since stopped doubting the output of our new overlords, LLM agents.

What I don’t understand is why he keeps having the agent build and run the benchmarks. There are hotkeys for this. Is the future of Visual Studio just a chat interface? Who is the target audience here?

Anyway, his new benchmark finds a problem with Boolean boxing issue and the profiler agent jumps on it, optimizing the code. He shows how tedious the stack trace would be to investigate—which is not tedious at all because he clicks through it quickly—but we’re also supposed to ignore how long that little progress circle next to “Analyzing performance trace” in the agent window is spinning. It takes long minutes while the developer has long since explained what the problem is and would likely have fixed it. The agent is really there for people who wouldn’t have understood the problem illustrated by the profiling trace and who wouldn’t be capable of judging the proposed solution.

The solution is wrong. He characterizes it as “the first time I ran it, it came up with a better solution,” but that’s a cop-out because the solution shown in the video doesn’t compile. He begs the agent to return a boolean instead of a string which, like, duh, because the whole problem was with boxing booleans. But, sure, let’s run the profiler by writing “run the benchmark again” in the chat window instead of hitting a f@&king hotkey. F@&k, people are absolutely in a cult about these agents!

“What’s really cool here is that the profiler agent was able to have a, um, successful impact on this code and help me contribute to this repository in a meaningful way when I don’t really know anything about this repository.”

WTF BRO.

You just made a video showing non-developers how to pad their GitHub commit histories with performance-improvement PRs that they don’t understand (and that might not even work) by spamming open-source projects.

I was more excited about this one, and I think it would have worked better without the agent, but he wanted to show the agent.

UIs

⛔ What’s New in Windows Forms by dotnet | Mary McGalla & Klaus Loeffelmann (YouTube)

The two presenters use a giant prompt with Copilot to build a .NET 10 Winforms app to show slides like PowerPoint. As usual, they feed this prompt in to the “planner” to get a more agent-friendly plan that they’ll send to the agent. They had to jabber quite a bit because the tool takes a long time to run.

The tool generates a list of steps in Markdown with checkboxes and a progress bar that it regenerates as it works. OK? I guess? Is Markdown a UI target now? WTF? Like, how shitty are your WPF or HTML skills when you’re hacking a new UI library on top of a Markdown renderer? Who thought that this was a good idea? I guess the last state of the UI is preserved and can be fed back in to the planner or agent?

It seems to have worked, though, … except that you can’t go to the next slide. Oh, no, wait, cursor keys are supported.

As usual, they didn’t show any of the content in the gigantic prompt that they wrote.

These two fools seem to have no idea how the tool that they spent 25 minutes using works.

Also, they barely talk about Winforms. The few things that they mentioned are better covered in the What’s new in Windows Forms for .NET 10 release notes.

This video sucked unless you enjoy watching people watch Visual Studio build code for them.

⛔ Modern Windows Development with .NET by dotnet | Roy & Michael Hawker (YouTube)

The two presenters discuss how much the community has done for WinUI3 development, with a huge style guide and much-better integration with the common MVVM toolkit also used in WPF and Maui. The WinUI3 styles can also be used with WPF, so that’s neat, I guess. They didn’t mention Maui. They talked about open-sourcing WinUI for quite a while.

They also pretty much watched Copilot do stuff like generating UI chunks from text examples, converting to JSON then to a view (I think). This was all running locally, on the NPU (Neural Processing Unit) rather than using a model in the cloud, which is kind of nice. However, it’s amazing how happy they are to demonstrate brute-forcing regeneration of a tool that generates a JSON then view from text, again and again and again.

No-one asks at all anymore whether the generated code is the same, whether it works, whether there are tests to verify it, whether it makes sense to generate umpteen copies, whether the time couldn’t be better spent on just doing it yourself, etc. etc. Of course, they never, ever show what was generated or give any indication that they have reviewed the code or consider it necessary to do so. Just run it once, look at it for a second, commit, push, and make a pull request.

Hey everyone! We’ve all been wasting our time all of these years with structured development practices. With this tool that’s right 70% of the time, you can skip all of that. Look at that UI go! Watch it flicker as it generates a whole bunch of stuff you’re never even going to bother looking at until you get a call at 03:00 in the morning because everything blew up. Just kidding. No-one’s going to call you. They’re going to call other people who were stupid enough to take jobs on an on-call team.

✅ TUIs Are Back (Although They Never Left): Creating Modern CLI Apps in .NET. by dotnet | Andres Pineda (YouTube)

He goes through the history of UIs for the first third of the video, which is kind of interesting and provides decent context for why we might want a TUI. In the second third, he presents the Spectre.Console framework for building TUIs. The initial version uses an in-memory database, then an SQLite database, and then an external database. It uses dependency injection and the by-now standard application startup.

He also discusses Terminal.Gui, which runs on all supported platforms and has Miguel de Icaza as a contributor. This one creates apps that kind of look the old Borland DOS-mode applications. You build them with MVVM (supports CommunityToolkit.Mvvm) and generated views (not XAML) that you build with a text-console-based visual designer. You kind of have to see it to believe it. It’s really pretty cool.

If you want to use XAML, though, you can use RazorConsole with Spectre.Console to build UIs with that instead.

🆗 Ship Faster with .NET MAUI: Real-World Pitfalls and How to Nuke Them by dotnet | Paul Usher (YouTube)

A lot of the pitfalls he discusses are relatively general: resolution, distribution, deployment, staying up to date with security, etc.

Dude recommends Console.WriteLine() as an important debugging tool. Ok, buddy. On the other hand, it’s nice to see someone who shows his whole setup in detail, which, even though some of his tools are outdated (e.g., he uses CodeRush!), is nice to see, especially if you really have no idea how to get started.

He goes on to discussing app-store-related problems and how to overcome some of them, which is also quite helpful, as this is a part of the process that few people talk about. It’s not particularly enlightening but it’s good to discuss, as you can’t deploy an app without getting on app store.

Another pitfall is dealing with lifecycle changes and interruptions: is the app in the foreground? Is the device asleep? Is there network connectivity? Is the battery low? Is the app in sleep mode? When do you perform which initialization? Which expectations can you have about connectivity? Everything is asynchronous and the situation outside the app changes all the time. You have to watch all of the events and respond appropriately.

He advises using the emulator or simulator for a tighter feedback loop but there’s no way to avoid testing on a target device—or multiple target devices, as their behavior varies as well. He mentions that two recent Android devices (a Pixel and a Samsung) had different behavior in crucial areas affecting his apps.

✅ Building Rock-Solid Avalonia Apps A Guide to Headless Testing with AI Assistance by dotnet | Dong Bin (YouTube)

Whereas Avalonia and Maui both support iOS, Android, Windows, and MacOS targets, Avalonia also support Linux targets, including Linux running on embedded systems. The target that Dong addresses though is the headless mode, which is used for end-to-end UI testing. Avalonia’s rendering is completely decoupled from the platform, with the headless platform being just another target, like Windows or Mac.

God bless him for actually showing us how to write tests in the code editor. he’s using Rider on Windows. His code uses ObservableProperty from the Community Toolkit. This is a good demo.

In an advanced demo, he shows how to use “screenshot” rendering, even in headless mode. He also shows how to test controls for performance, both in speed and memory-usage, which is very important for building controls for highly constrained environments like embedded systems.

He points out that headless testing won’t help you with testing native features, actual visual look-&-feel. Instead, you can use the Skia renderer to approximate tests like that.

Finally, he actually introduces a usage of AI that makes sense to me: helping to write all of the unit, integrated, headless, and render tests. He explains how the task is focused, verifiable, and already has a lot of context to keep the generated code on the right path.

✅ What’s New in .NET MAUI by dotnet | David Ortinau (YouTube)

This one starts with an overview of the project. SyncFusion contributes heavily, from dozens of PRs to providing over 30 controls as open-source controls.

They’re also working much more closely with the Uno platform, which is ostensibly a competing framework but seems to be merging or moving closer to Maui. They’re working on NativeAOT for Android, SkiaSharp improvements (it’s their main rendering library), as well as WebAssembly multi-threading (that’s another target that they have that Maui does not, unless you count Blazor integration).

His demonstration is kind of neat: he shows a Maui app with SyncFusion controls and Community Toolkit, all running within an Uno Platform App. He shows it running in an Android emulator. This kind of support may extend Maui’s reach without having to replicate everything. For example, the WebAssembly target Uno offers works seamlessly with .NET Maui apps. He demos a NuGet browser that was written for desktop, but now running in a browser.

Next up is a very prosaic but very welcome addition: global usings/namespace declarations for XAML files. You no longer need to use prefixes and you no longer have a clump of stuff at the top of the file. On top of that, they also now support implicit namespaces (the feature is in preview).

Now a XAML file for Maui can look like this:

<ContentPage x:Class="DeveloperBalance.Pages.MainPage"
             x:DataType="MainPageModel"
             x:Name= "OverviewPage"
             Title="{Binding Today}">

This is really nice.

There’s also XAML source-generation now. This increases speed of debugging and reduces the differences between the debug and release builds massively. This is an opt-in feature but it sounds great. You can debug the generated code instead of relying on a bunch of reflection. Debugging uses 99% less memory and view-inflation is now 1000% faster (10x). Overall app performance is 25% faster with 30% less memory usage.

He talks about support for “safe edges” (UI integration with mobile form factors) and improved support for hybrid apps. He briefly discusses Aspire orchestration, which is completely integrated. This is especially interesting with hybrid solutions because the front-end actually has two parts that need to be coordinated. Doing this with Aspire is interesting. You can use the dashboard to inspect telemetry because the standard rendering is integrated as well. This telemetry is also available on the command line if you don’t use Aspire.

✅ Build better web apps with Blazor in .NET 10 by dotnet | Daniel Roth (YouTube)

The author talks a bit about large-scale apps in the U.S. and Europe that are built with Maui and, specifically, Maui Blazor. His presentation in this part is quite stilted and seems to have been massaged by the PR department. Like, he says that .NET Aspire makes you “cloud-ready,” which, if you’ve watched the Aspire talks, is no longer the focus of Aspire, and hasn’t been for a while. Deploying to the cloud is possible and well-supported, but it’s not the main use case.

He does demo some code, though. He shows passkey-integration for Blazor apps. I love how people watch this and think, “this is great; so much easier to log in,” whereas I watch it and have just watched someone log in using a 4-digit PIN rather than a safe password. How is this better? It’s similar to using a password manager on your device that’s always logged in, though. But passkeys are really replicating a bunch of the convenience that you already had with a password manager.

Next up is better integration for telemetry, which all appears in the Aspire dashboard. There are also advanced diagnostics, like being able to extract memory dumps and low-level runtime metrics from a running WASM Blazor app using a JavaScript command. The dottrace file can be easily converted to a gcdump file using the dotnet command and can then be analyzed in Visual Studio. This got very technical very quickly and I am here for it.

Blazor is also about 20% faster in .NET 10. For developers,

• Hot Reload is better; he demonstrates an over 10x speed improvement, from 38s to about 3s.
• Full-graph form-validation, so complex forms no longer need custom validation.
• Automated browser/end-to-end testing using WebApplicationFactory but then also launching a full-fledged headless browser and then running Playwright tests against it.
• Better state-persistence support, with automatic persistence on idle, pause/resume on idle, etc. This all integrates with the telemetry and can be inspected in the Aspire dashboard.

Very interesting and encouraging.

Aspire

✅ Taking .NET out of .NET Aspire − working with non-.NET applications by dotnet | David Gardiner (YouTube)

He presents a multi-language, multi-environment solution that uses Python/uv, Rust/cargo, and TypeScript/pnpm, each of which are run manually. From there, he shows a template Aspire solution with a Redis cache, an API service, and a web front-end.

He starts with a new Aspire solution, then integrates Mongo support using aspire-add-mongo and then integrates the PowerShell script that populates the data using an Aspire API. With that loaded up, he searches for an Aspire extension that works with his existing Python/uv setup. He doesn’t have to change anything; he just binds the startup of that part into Aspire so that the service is available to his “app host” (and also shows up on the dashboard). The Rust service easily follows, again by using an existing Aspire package to integrate Rust/cargo specifically. Finally, he binds the React/Vite/pnpm solution using a node.js extension from the Community Toolkit (again).

Where Aspire shines is that you don’t need to run these disparate apps from various command lines or scripts, and you don’t need to configure containers with YAML; you bind the various components and services with C# code, indicating dependencies between them, which Aspire not only handles but displays in the dashboard.

He uses this power to remove hard-coded ports from his services, using the C# variables to read the and use the dynamically assigned ports instead. Finally, he integrates OpenTelemetry into the Python and Rust services so that the various services show their telemetry in the Aspire console, structured logging, traces, and metrics views.

Finally, he adds an extra service that uses a node backend. Adding it once you have Aspire configured is very, very easy.

This is an absolutely great 22-minute video that you can send to anyone who asks “what can Aspire do for me?”

⛔ From Architecture to Docs: .NET Aspire Documented with Copilot by dotnet | Jorge Fernandez & David Oliva (YouTube)

This video explains the basics of Aspire (like, the very basics), as well as the basics of Copilot and MCP. You can skip that part, as they’re just reading from the slides, in what I am forced to note are pretty strong Spanish accents.

I honestly can barely tell what’s going on here. I feel so bad for these guys because they are probably much better in their native language but it’s so much work understanding them in English. They’re generating stuff with Copilot to generate an architecture overview for an existing solution file, using Markdown and ASCII diagrams. They then upgrade to using Mermaid diagrams. But I dare you to replicate what they did.

✅ Windows 365 Meets Aspire − Supercharging Multi-Repo Microservice Productivity by dotnet | Eric Guo & Chuanbo Zhang (YouTube)

This video demonstrates using .NET Aspire to wire up microservice servers with simulated Azure services in order to test InTune deployment software. It’s quite a complex use case. They show how you can test locally, using Docker and the Azure-service simulators, and also deploy to Azure infrastructure.

They even show how to simulate some of your own microservices by using the VS .http file format to quickly mock responses for a subset of the functionality. In this vein, they also discuss how to configure data-seeding for a stable environment, then finish up by discussing how to use XUnit to run automated tests against this entire infrastructure, both locally and in pipelines.

Although the specific use case is quite complex, there is a lot of good stuff to learn about testing automation in this talk. .NET Aspire makes it a lot easier to run locally and in the cloud without different approaches.

✅ Deep Dive: Extending and Customizing Aspire by dotnet | David Fowler & Damian Edwards (YouTube)

Fowler shows a single-project solution with a .NET Aspire AppHost project that binds non-.NET dependencies (i.e., they’re not they’re own projects). One of the dependencies is a postgres database that is absolutely a dependency but has classically been managed outside of the solution. Now, you can declare and bind the dependencies with C#. The takeaway is: a much slimmer readme file, that you just clone and call aspire run.

The great thing about this is that it has to stay in-sync, unlike a readme file.

Fowler shows the app dashboard with a lot of custom dependencies, including the .NET 10 OpenAPI replacement called Scalar, which is fully integrated into the Aspire dashboard. Fowler even shows how you can customize the dashboard appearance with C# code, using very standard options customization, as you would see in other host-based applications like ASP.Net (or many other types, Console, Windows Service, etc.).

Damian points out what we’re all thinking: holy crap, Fowler, WTH you hacked everything into the AppHost.cs file, like hundreds of lines, including a custom database seeder that uses the endpoint spun up by Aspire. It’s neat to see how you can bind in that kind of code, though, to just wait until the HTTP REST server is available and then to run some C# code to seed it with data. It’s ugly and it’s hacky in his code, but it’s wonderful that you can prototype and test so quickly with disparate systems and components. He has only one C# file and orchestrates diverse other components and scripts from it.

OK, he continues to show how you can bind commands into the Aspire Dashboard that he uses to bind a “reset command” that uses the Aspire interaction service to show a message box requesting approval.

Finally, at the very end, he shows how to use an MCP integration with Aspire. This is no more exciting than watching anyone else watch Copilot stumble drunkenly around a dark room. It’s only the last two minutes so we’re not subjected to too much of this foolishness. It was still writing furiously into the chat as the video ended.

Fowler is also using Visual Studio Code rather than Visual Studio. He also speaks very, very quickly, so brace yourself.

✅ Aspire Unplugged with David and Maddy by dotnet | David Fowler & Maddy Montaquila (YouTube)

They have T-Shirts with a great sentiment on them, “Friends don’t let friends write YAML.” Except that the “Write YAML” part is really, really big for some reason, so it looks like the shirts are exhorting users to actually write YAML. Whatever.

The first question is for Fowler, who describes the impetus of Aspire. It came from the pains of configuring so many scripts for infrastructure, even with a strong tool like Kubernetes.

It grew into a “general-purpose dev tool” for any sort of environment. It was originally scoped as a cloud-native tool but it quickly became obvious that nearly every solution has some sort of orchestration and scripting that always ended up in readme files or PowerShell or Bash scripts: starting the database, starting the backend for a mobile app, whatever.

“That became one of our key things, right? Like you want to onboard someone, you model all the stuff in code and then like you don’t have to tell someone run this script, run that script, pass the output from this script to that script, string together stuff. Like you can just kind of like put it in code, have it be there.”

He gives a lot of examples and detail about how polyglot and scalable .NET Aspire is. The other video he did—Deep Dive: Extending and Customizing Aspire by dotnet | David Fowler & Damian Edwards (YouTube)—showed a lot of code for integrating JavaScript and Python services. Another video—Taking .NET out of .NET Aspire − working with non-.NET applications by dotnet | David Gardiner (YouTube)—also shows how to integrate a lot of plugins from the community, including a Rust backend service.

The next big question is about persisting containers, supporting hot-reload, which is finicky to design and increases the complexity of the architecture significantly but the upside is huge if they can get it working. They managed a huge rewrite of all of the plumbing to support this type of scenario and are much better positioned for future developments.

The next question builds on this, asking about multi-repo support, with what’s called the “AppHost in AppHost” question: can you nest .NET Aspire apps? How does that work? It would be nice to be able to group shared services into one AppHost and then reference then from another high-level AppHost (for much larger solutions, obviously). What happens to the dashboards, though?

The idea of Aspire is to work with existing solutions, so the aspire init is a much more important workflow than aspire new. That is, you’re much more likely to already have a solution into which you’d like to integrate an AppHost or set of projects around which you’d like to wrap an AppHost than you are to be green-fielding a solution and starting with Aspire.

I love the dynamic between Fowler and Maddy. You can really tell they love working together, that they really, really respect one another. They love the “adult” Damian as well.

Cloud

🆗 What’s New in Containers for .NET 10 by dotnet | Rich Lander & Chet Husk (YouTube)

The two presenters first discuss the history of containers in .NET, including operating systems, support periods, etc. The second half demonstrates using dotnet publish using AOT and multiple OS targets and then deploying them into various containers. This targets are all variations of Linux and for command-line or server apps.

✅ What’s new in Azure App Service for .NET developers by dotnet | Byron Tardif (YouTube)

He quickly covers when .NET 10 will be available in App Service for Linux (Ubuntu, not Debian) and Windows, then moves on to showing how to use .NET Aspire to build and deploy an application to App Service.

Blessedly, he’s doing it manually, following a simple guide, rather than “getting Copilot to do it for him.” This inspires much more confidence that it’s well-designed and simple enough to actually learn, rather than implying that you need to ask a black-box globe-girdling data-model in order to grok it.

He’s got the standard Aspire app and then types azd up. It takes five minutes for the system to analyze, find a subscription, determine existing resources, and then deploy, creating services where needed. Access to the deployment is automatically configured (e.g., the dashboard is only available for authorized users).

He quickly shows the Azure Portal resources that were created for the App Service. This is nice. .NET Aspire is a worthy and welcome successor to Bicep scripts.

He shows a bunch of features of App Services specifically, including scaling options.

🆗 Carbon Aware Computing − Using .NET Open Source libraries for more sustainable applications by dotnet | Aydin Mir Mohammadi (YouTube)

This video covers tactics and tooling for running data services in a sustainable manner. E.g., load-shifting from day to night, adjusting available capacity depending on local energy availability, etc. There’s a lot of telemetry and real-time monitoring needed to even begin working in a sustainable manner.

In the second half, he gets to integrating an SDK that calculated best-execution time. Even libraries like Hangfire have methods like IncludeCarbonAwareExecution() (I’m not kidding!) that wrap all of this in a very high-level abstraction.

AI

⛔️ Architecting an AI-Powered Sales Dashboard with .NET MAUI and Azure OpenAI by dotnet | Shriram Sankaran (YouTube)

The app he discusses summarizes market data using AI. Did we all just choose to forget that AIs are not good at numbers? Did I miss the technology that we used to fix this problem? Remember “AIs are not good at numbers?” I do! When did we fix that?

Anyway, the UI looks decent and it’s completely cross-platform thanks to Maui. It uses SyncFusion’s controls as well as standard Maui controls. He spends quite a bit of time going over the features of his app. The AI is used to query the app data with a built-in chatbot.

When he finally gets to the code, his project is curiously not using CommunityToolkit.MVVM (all of the properties are implemented manually instead of source-generated. He eventually gets to more source but it’s not very illuminating. I can’t really recommend it.

🆗 One Question, One Answer: Designing Seamless AI Agents with C# by dotnet | Mark Miller (YouTube)

The presenter works on CodeRush for DevExpress. He uses CodeRush (I guess?) in dictation mode to build his calculator app, which, you know, is going to be something that the AI can easily build, as there are probably millions of examples in the training data. The generated code is horrifically defensive and not even close to what I would have made, or what I consider to be maintainable, but it’s fine for a prototype.

So, here we have another video that’s just showing how to program with an AI. He’s arguing for a workflow that stays in the code and is delivered via talking—because it’s 2-4 times faster than typing for most people and LLMs are very forgiving of extra words and filler words—so that you can avoid most of the pain points of working with the by-now “classic” AI-chat interface.

He talks about lot about how to optimize the context but I guess his tool does this?

✅ Overcoming the limitations when using AI by dotnet | Michael Washington (YouTube)

This guy doesn’t show up on the video. His voiceover and cadence is somewhat odd. It sounds very much like a text-to-speech engine. The whole presentation seems fake but the information is quite interesting. I guess he wrote the presentation but then had a machine read it for him.

He discusses how LLMs are bad at math, so the solution was to have the LLM create code to calculate answers. It’s wild how much f@&king processing power we’re willing to invest in getting the correct answer to 43 × 34. The LLM interprets the text, then generates an answer that includes a little Python program that it then executes in a sandbox so that i can include the output in its answer. It’s just flat-out nuts. Still, he shows off how he’s managed to work around these limitations but they are really elaborate.

Next up is that “AIs can’t write fiction”. He discusses AI story-builders, which use text-file databases in order to maintain context and continuity for stories. He found that page-by-page and chapter-by-chapter doesn’t work very well, but that paragraph-by-paragraph is the level of granularity at which an LLM needs guidance. There is a whole program surrounding the LLM’s inputs and outputs. Without it, the story goes off the rails immediately.

After that, he shows that AI cannot create applications. They can code but they have no idea of architecture and no idea how to deal with complex systems.

Find his slides and work at Overcoming limitations When Using AI.

🆗 Modernizing a 17th Century Italian-English Dictionary by dotnet | Wayne Sebbens (YouTube)

This was not uninteresting but it wasn’t a lot of programming information. Half of the video is a discussion of European martial arts and its relation to archaic Italian dialects and spellings. He basically made an app for searching these terms using vector databases and ML in .NET. If that sounds like something you want to do, check out the video and his repo (GitHub). If not, then you can safely skip the video.

Everything else

🆗 Going Passwordless − A Practical Guide to Passkeys in ASP.NET Core by dotnet | Maarten Balliauw (YouTube)

This is a decent and thorough introduction to authentication mechanisms, from passwords to MFA to passkeys, illustrating both the differences between passkeys and other methods as well as the .NET support for working with passkeys in your own applications (mostly in the last third of the video).

✅ GitHub Actions DevOps Pipelines as Code using C# and Cake SDK by dotnet | Mattias Karlsson (YouTube)

Cake is a build system written in C# with a rich .NET API. Mattias did a bunch of live-coding. The Cake scripts might be useful for defining a bunch of stuff that we currently use Azure Pipeline Definitions for. he demonstrates how provider plugins enable high-level abstractions that make it much easier to specify a declarative pipeline. It’s all in C#, so you use a code editor like Rider, with code-completion, refactoring, etc.

You continue to use the YAML pipeline definition to set up the environment but everything else will be in the Cake file. This makes a lot of sense and could be quite powerful. Instead of using a bunch of pipeline nested templates that you can’t run or debug, you could have a NuGet package with common APIs for Cake. You can also test a bunch of the Cake script locally (unless you have some highly specific steps like signing with a key only available in the cloud or calling a tool that’s only available in the cloud. You can use standard C# to make these optional when testing locally, though.

🆗 If .NET brewed beer… by dotnet | Shaun Lawrence (YouTube)

He starts with a 10-minute presentation on his home-brewing setup, finally getting to the point where he discusses the embedded device for which he used .NET: A Meadow F7v2 DevModule. For the next ten minutes, he just kind of muddles about, showing the API surface of the meadow library.

After showing how to integrate a temperature sensor, he shows how to integrate PID control (Proportional-Integral-Derivative control), again using the API. He mixes in support for PWN (Pulse-width Modulation). Both of these are commonly used algorithms to stabilize the interaction with a sensor: for interpreting and smoothing the signal and for ensuring that the written value corresponds to the desired value without slewing about. At the very end, he shows that his UI is built with Maui but he doesn’t get into it too much.

It’s nice that they provide low-level support for working directly with hardware but it’s not too fascinating. It’s good to know that C# is increasingly becoming a viable alternative to systems programming with C, C++, or even Rust or Go. He uses Visual Studio Code.

The book-length Performance Improvements in .NET 10 by Stephen Toub (Microsoft DevBlogs) arrived a couple of months ago.

He explains how the various compilers (AOT, JIT, etc.) have been optimized to eliminate allocations and just generally optimized for performance. A reduction in allocations is a multi-win: the performance is better because the allocator isn’t working, the memory usage has dropped, and the garbage collector also works less.

See previous coverage in:

• Toub’s 234-page tour-de-force on performance in .NET 9 (2024)
• Somehow, I never documented .NET 8. Huh.
• Performance Improvements in .NET 7 (2022).

A presentation at .NET Build 2025

If you prefer a 30-minute video, then you’re in luck.

Performance Improvements in .NET 10 by dotnet | Stephen Toub (YouTube)

He compares .NET Framework 4.8 vs. .NET 9 vs. .NET 10. The most impressive improvements are from 4.8 to 9.0, of course, but he highlights some interesting places where .NET 10 eclipses .NET 9, where .NET 9 had already eclipsed .NET Framework 4.8.

The last example shows how regular expressions have been continually optimized so that an operation that took 24ms in .NET Framework 4.8 was improved by about 12x to 2.5ms in .NET 9 but has been further improved by about 62,500x to about 40ns in .NET 10.

Citations and Notes

And now, on to the citations from Toub’s book along with my notes.

He starts off with a bit of history and context in the wider world.

“What made Tudor’s ice last halfway around the world wasn’t one big idea. It was a plethora of small improvements, each multiplying the effect of the last. In software development, the same principle holds: big leaps forward in performance rarely come from a single sweeping change, rather from hundreds or thousands of targeted optimizations that compound into something transformative. .NET 10’s performance story isn’t about one Disney-esque magical idea; it’s about carefully shaving off nanoseconds here and tens of bytes there, streamlining operations that are executed trillions of times.”

“As with many languages, .NET historically has had an “abstraction penalty,” those extra allocations and indirections that can occur when using high-level language features like interfaces, iterators, and delegates. Each year, the JIT gets better and better at optimizing away layers of abstraction, so that developers get to write simple code and still get great performance. .NET 10 continues this tradition. The result is that idiomatic C# (using interfaces, foreach loops, lambdas, etc.) runs even closer to the raw speed of meticulously crafted and hand-tuned code.”

JIT

“If the compiler can prove an object doesn’t escape, then that object’s lifetime is bounded by the method, and it can be allocated on the stack instead of on the heap. Stack allocation is much cheaper (just pointer bumping for allocation and automatic freeing when the method exits) and reduces GC pressure because, well, the object doesn’t need to be tracked by the GC. .NET 9 had already introduced some limited escape analysis and stack allocation support; .NET 10 takes this significantly further.”

“[…] where things gets interesting is around what the JIT is able to devirtualize. In .NET 9, it struggles to devirtualize calls to the interface implementations specifically on T[], so it won’t devirtualize either the _list.GetEnumerator() call nor the _list[index] call. However, the enumerator that’s returned is just a normal type that implements IEnumerator<T>, and the JIT has no problem devirtualizing its MoveNext and Current members. Which means that we’re actually paying a lot more going through the indexer, because for N elements, we’re having to make N interface calls, whereas with the enumerator, we only need the one with GetEnumerator interface call and then no more after that.”

To be clear: this has been addressed in .NET 10, so that the indexer is also almost always devirtualized.

“dotnet/runtime#110827 from @hez2010 also helps more methods to be inlined by doing another pass looking for opportunities after later phases of devirtualization. The JIT’s optimizations are split up into multiple phases; each phase can make improvements, and those improvements can expose additional opportunities. If those opportunities would only be capitalized on by a phase that already ran, they can be missed. But for phases that are relatively cheap to perform, such as doing a pass looking for additional inlining opportunities, those phases can be repeated once enough other optimization has happened that it’s likely productive to do so again.”

“The static readonly field is immutable, arrays can’t be resized, and the JIT can guarantee that the field is initialized prior to generating the code for Read. Therefore, when generating the code for Read, it can know with certainty that the array is of length three, and we’re accessing the element at index two. Therefore, the specified array index is guaranteed to be within bounds, and there’s no need for a bounds check.”

The JIT has been doing these kinds of optimizations for a long time but the number of cases for which it can “prove” increases with each release.

“My choice of benchmark in this case was not coincidental. This pattern shows up in the FormattingHelpers.CountDigits internal method that’s used by the core primitive types in their ToString and TryFormat implementations, in order to determine how much space will be needed to store rendered digits for a number. As with the previous example, this routine is considered core enough that it was using unsafe code to avoid the bounds check. With this fix, the code was able to be changed back to using a simple span access, and even with the simpler code, it’s now also faster.”

“Many of these different optimizations interact with each other. Dynamic PGO triggers a form of cloning, as part of the guarded devirtualization (GDV) mentioned earlier: if the instrumentation data reveals that a particular virtual call is generally performed on an instance of a specific type, the JIT can clone the resulting code into one path specific to that type and another path that handles any type. That then enables the specific-type code path to devirtualize the call and possibly inline it. And if it inlines it, that then provides more opportunities for the JIT to see that an object doesn’t escape, and potentially stack allocate it. dotnet/runtime#111473, dotnet/runtime#116978, dotnet/runtime#116992, dotnet/runtime#117222, and dotnet/runtime#117295 enable that, enhancing escape analysis to determine if an object only escapes when such a generated type test fails (when the target object isn’t of the expected common type).”

This led to several several dozen performance-test improvements across the board when the PR landed. The whole section boils down to the JIT optimization working not only for regular loops, enumerable loops, but also hand-unrolled code with multiple array accesses (where bounds-checks can now be elided using clever cloning).

Inlining

“[…] generally the most benefit from inlining comes from knock-on benefits. Just as a simple example, if you have code like:”

int i = Divide(10, 5);

static int Divide(int n, int d) => n / d;

“if Divide doesn’t get inlined, then when Divide is called, it’ll need to perform the actual idiv, which is a relatively expensive operation. In contrast, if Divide is inlined, then the call site becomes:”

int i = 10 / 5;

“which can be evaluated at compile time and becomes just:”

int i = 2;

“Just inlining everything would be bad; inlining copies code, which results in more code, which can have significant negative repercussions. For example, inlining’s increased code size puts more pressure on caches. Processors have an instruction cache, a small amount of super fast memory in a CPU that stores recently used instructions, making them really fast to access again the next time they’re needed (such as the next iteration through a loop, or the next time that same function is called).”

“As part of these heuristics, the JIT has the notion of “boosts,” where observations it makes about things methods do boost the chances of that method being inlined. dotnet/runtime#114806 gives a boost to methods that appear to be returning new arrays of a small, fixed length; if those arrays can instead be allocated in the caller’s frame, the JIT might then be able to discover they don’t escape and enable them to be stack allocated. dotnet/runtime#110596 similarly looks for boxing, as the caller could possibly instead avoid the box entirely.”

Code Layout

“When the JIT compiler generates assembly from the IL emitted by the C# compiler, it organizes that code into “basic blocks,” a sequence of instructions with one entry point and one exit point, no jumps inside, no branches out except at the end. These blocks can then be moved around as a unit, and the order in which these blocks are placed in memory is referred to as “code layout” or “basic block layout.” This ordering can have a significant performance impact because modern CPUs rely heavily on an instruction cache and on branch prediction to keep things moving fast. If frequently executed (“hot”) blocks are close together and follow a common execution path, the CPU can execute them with fewer cache misses and fewer mispredicted jumps.”

“Consider a tight loop executed millions of times. A good layout keeps the loop entry, body, and backward edge (the jump back to the beginning of the body to do the next iteration) right next to each other, letting the CPU fetch them straight from the cache. In a bad layout, that loop might be interwoven with unrelated cold blocks (say, a catch block for a try in the loop), forcing the CPU to load instructions from different places and disrupting the flow. Similarly, for an if block, the likely path should generally be the next block so no jump is required, with the unlikely branch behind a short jump away, as that better aligns with the sensibilities of branch predictors.”

GC Write Barriers

“Whenever there’s a reference write that could cross a generation, the JIT emits a call to a helper that tracks the information in a “card table,” and when the GC runs, it consults this table to see if it needs to scan a portion of the higher generations. That helper is referred to as a “GC write barrier.” Since a write barrier is potentially employed on every reference write, it must be super fast, and in fact the runtime has several different variations of write barriers so that the JIT can pick one optimized for the given situation. Of course, the fastest write barrier is one that doesn’t need to exist at all, so as with bounds checks, the JIT also exerts energy to try to prove when write barriers aren’t needed, eliding them when it can. And it can even more in .NET 10.”

Miscellaneous

“As with most compilers, the JIT employs common subexpression elimination (CSE) to find identical computations and avoid doing them repeatedly. dotnet/runtime#106637 teaches the JIT how to do so in a more consistent manner by more fully integrating CSE with its Static Single Assignment (SSA) representation. This in turn allows for more optimizations to kick in, e.g. some of the strength reduction done around loop induction variables in .NET 9 wasn’t applying as much as it should have, and now it will.”

I just love how Toub manages to keep up his excitement so deep into this document. He’s really a great writer.

Native AOT

“Native AOT [Ahead Of Time [compilation]] is the ability for a .NET application to be compiled directly to assembly code at build-time. The JIT is still used for code generation, but only at build time; the JIT isn’t part of the shipping app at all, and no code generation is performed at run-time. As such, most of the optimizations to the JIT already discussed, as well as optimizations throughput the rest of this post, apply to Native AOT equally.”

VM

“With dotnet/runtime#114462, the runtime now uses a single shared “template” for many of the small executable “stubs” it needs at runtime; stubs are tiny chunks of machine code that act as jump points, call counters, or patchable trampolines. Previously, each memory allocation for stubs would regenerate the same instructions over and over. The new approach builds one copy of the stub code in a read-only page and then maps that same physical page into every place it’s needed, while giving each allocation its own writable page for the per-stub data that changes at runtime. This lets hundreds of virtual stub pages all point to one physical code page, cutting memory use, reducing startup work, and improving instruction cache locality.”

Threading

“If a thread is blocked on an operation that depends on work items in that thread’s local queue getting processed, that work item being picked off now depends on the global queue being exhausted and another thread coming along and stealing the work item from this thread’s queue. If there’s a steady stream of incoming work into the global queue, though, that will never happen; essentially, the highest priority work item has become the lowest priority work item.

“So, back to these PRs. The idea is fairly simple: when the thread is about to block, and in particular when it’s about to block waiting on a Task, it first dumps its entire local queue into the global queue. That way, this work which was highest priority for the blocked thread has a fairer chance of being processed by other threads, rather than it being the lowest priority work for everyone.”

“dotnet/runtime#107843 from @hamarb123 adds two new methods to the Volatile class: ReadBarrier and WriteBarrier. A read barrier has “load acquire” semantics, and is sometimes referred to as a “downward fence”: it prevents instructions from being reordered in such a way that memory accesses below/after the barrier move to above/before it. In contrast, a write barrier has “store release” semantics, and is sometimes referred to as an “upwards fence”: it prevents instructions from being reordered in such a way that memory accesses above/before the barrier move to below/after it.”

“These barriers are referred to as “half fences”; the read barrier prevents later things from moving earlier, but not the other way around, and the write barrier prevents earlier things from moving later, but not the other way around. (As it happens, though, while not required by specification, today the implementation of lock does use a full barrier on both enter and exit, so nothing before or after a lock will move into it.)”

Reflection

“System.Net.Http sits above System.Security.Cryptography, referencing it for critical features like X509Certificate. But System.Security.Cryptography needs to be able to make HTTP requests in order to download OCSP information, and with System.Net.Http referencing System.Security.Cryptography, System.Security.Cryptography can’t in turn explicitly reference System.Net.Http. It can, however, use reflection or [UnsafeAccessor] and [UnsafeAccessorType] to do so, and it does. It used to use reflection, now in .NET 10 it uses [UnsafeAccessor].”

Primitives and Numerics

“dotnet/runtime#111505 from @alexcovington enables TensorPrimitives.Divide<T> to be vectorized for int. The operation already supported vectorization for float and double, for which there’s SIMD hardware-accelerated support for division, but it didn’t support int, which lacks SIMD hardware-accelerated support. This PR teaches the JIT how to emulate SIMD integer division, by converting the ints to doubles, doing double division, and then converting back.”

That fix, roundabout as it sounds, ends up making that operation 4x faster. This is pretty cool because dividing integers in SIMD code just became 4x faster on .NET. You don’t use this, you say? Well, are you sure? Are you sure that there is no code in handshake-negotiation (e.g.) that needs to divide multiple integers in parallel? These are exactly the kind of improvements that, as noted in Toub’s introduction, lead to smoother operation in many other places. This is such a low-level primitive.

“ We can then reuse those methods to do the same thing that’s already done for scalar operations but do it vectorized: take a vector of Halfs, convert them all to floats, process all the floats, and convert them all back to Halfs. Of course, I already stated that the vector types don’t support Half, so how can we “take a vector of Half“? By reinterpret casting the Span<Half> to Span<short> (or Span<ushort>), which allows us to smuggle the Halfs through. And, as it turns out, even for scalar, the very first thing Half‘s float cast operator does is convert it to a short.

“The net result is that a ton of operations can now be accelerated for Half.”

These optimizations improve performance for processing Half in dozens of operations by 11x.

“with C# 14, it’s possible for a type to not only define a + operator, it can also define a += operator. If a type defines a += operator, it will be used rather than expanding a += b as shorthand for a = a + b. And that has performance ramifications.

“[…] that means that such compound operators on the tensor types can just update the target tensor in place rather than allocating a whole new (possibly very large) data structure for each computation. dotnet/runtime#117997 adds all of these compound operators for the tensor types. (Not only are these using C# 14 user-defined compound operators, they’re doing so as extension operators, using the new C# 14 extension types feature. Fun!)”

Collections

“[…] as noted earlier in the JIT section, the JIT has been gaining super powers around dynamic PGO, escape analysis, and stack allocation. This means that in many situations, the JIT is now able to see that the most common concrete type for a given call site is a specific enumerator type and generate code specific to when it is that type, devirtualizing the calls, possibly inlining them, and then, if it’s able to do so sufficiently, stack allocating the enumerator. With the progress that’s been made in .NET 10, this now happens very frequently for arrays and List<T>. While the JIT is able to do this in general regardless of an object’s type, the ubiquity of enumeration makes it all that much more important for IEnumerator<T>, so dotnet/runtime#116978 marks IEnumerator<T> as an [Intrinsic], giving the JIT the ability to better reason about it.”

“For shorter lists, dynamic PGO will see MoveNextRare invoked a reasonable number of times, and will consider it for inlining. And if all of the calls to the enumerator are inlined, the enumerator instance can avoid escaping the call frame, and can then be stack allocated. But once the list length grows to a much larger amount, that MoveNextRare method will start to look really cold, will struggle to be inlined, and will then allow the enumerator instance to escape, preventing it from being stack allocated.”

“While OSR is awesome, it unfortunately causes some complications here. Once the list gets long enough, an invocation of the tier 0 (unoptimized) method will transition to the OSR optimized method… but OSR methods don’t contain dynamic PGO instrumentation (they used to, but it was removed because it led to problems if the instrumented code never got recompiled again and thus suffered regressions due to forever-more running with the instrumentation probes in place). Without the instrumentation, and in particular without the instrumentation for the tail portion of the method (where the enumerator’s Dispose method is invoked), even though List<T>.Dispose is a nop, the JIT may not be able to do the guarded devirtualization that enables the IEnumerator<T>.Dispose to be devirtualized and inlined. Meaning, ironically, that the nop Dispose causes escape analysis to see the enumerator instance escape, such that it can’t be stack allocated. Whew.

“[…] Specifically for enumerators, this PR enables dynamic PGO to infer the missing instrumentation based on the earlier probes used with the other enumerator methods, which then enables it to successfully devirtualize and inline Dispose.”

“Labels A and B form a loop, but that loop can be entered by jumping to either A or to B. If the compiler could prove that this loop were only ever enterable from A or only ever enterable from B, then the loop would be “reducible.” Irreducible loops are much more complex than reducible loops for a compiler to deal with, as they have more complex control and data flow and in general are harder to analyze. dotnet/runtime#116949 rewrites the MoveNext method to be a more typical while loop, which is not only easier to read and maintain, it’s also reducible and more efficient, and because it’s more streamlined, it’s also inlineable and enables possible stack allocation.”

This results in a 7x performance improvement when iterating a list of integers.

There are also a ton of optimizations in Linq, for Contains (with 10x − 400x improvements), Fill (40x), Shuffle (2x − 40x), LeftJoin, and RightJoin (2x). There are also specific improvements for many of the base collection types.

IO

The next section on IO is also interesting, with one case where they didn’t actually change any code but instead introduced an analyzer that discourages using the EndOfStream property in asynchronous code, which can lead to pathological cases in which the stream is blocked until more data arrives.

Searching / Regular Expressions

This section includes a longer discussion about the improvements included in previous versions of .NET, especially as it relates to avoiding backtracking. There are normalized forms of regular expressions that incur no backtracking penalty and can thus be evaluated with the faster version of the regular-expression engine that doesn’t have to account for it.

Here’s an example that I’ve lifted up from much further down in this section.

“Given the pattern ^abc|^abd, the code generators would end up emitting this exactly as it’s written, with an alternation with two branches, the first branch checking for the beginning and then matching “abc”, the second branch also checking for the beginning and then matching “abd”. Now in .NET 10, the anchor can be factored out, such that ^abc|^abd ends up being rewritten as ^ab[cd].”

The idea here is to search for pathological formulations for which there is a non-pathological equivalent and automatically use that version under the hood. That is my interpretation of the following rather-dense section.

“Consider a pattern a*b. a*b is observably identical to (?>a*)b, which says that the a* should not be backtracked into. That’s because there’s nothing the a* can “give back” (which can only be as) that would satisfy what comes next in the pattern (which is only b). It’s thus valid for a backtracking engine to transform how it processes a*b to instead be the equivalent of how it processes (?>a*)b. And the .NET regex engine has been capable of such transformations since .NET 5. This can result in massive improvements to throughput. With backtracking, waving my hands, we effectively need to execute everything after the backtracking construct for each possible position we could backtrack to. So, for example, with \w*SOMEPATTERN, if the w* successfully initially consumes 100 characters, we then possibly need to try to match SOMEPATTERN up to 100 different times, as we may need to backtrack up to 100 times and re-evaluate SOMEPATTERN each time we give back one of the things initially matched. If we instead make that (?>\w*), we eliminate all but one of those! That makes improvements to this ability to automatically transform backtracking constructs to be non-backtracking possibly massive improvements in performance, and practically every release of .NET since .NET 5 has increased the set of patterns that are automatically transformed. .NET 10 included.”

There are several detailed examples of 5x–6x improvements in performance for relatively common-looking regular expressions. Stephen Toub loves writing about very-specific regular-expression examples. Like, each paragraph is a blog post just on its own. Needless to say, this section is, at the same time, fascinating, extremely detailed, and eminently uncitable (because it would just entail citing pages of detail that is all necessary to understand the optimization). The improvements are impressive and incredibly well-described. Go check out that section if you like regular expressions and mathematical analysis (equivalence of expressions, reduction of solution space). The additional beauty is that the regular-expression evaluators are all source-generated C#, so it’s much, much easier to evaluate what’s going on than with the assembly-level discussions in the JIT discussion, for example.

As a final example, here is the level of holistic analysis we’re talking about.

“Unfortunately, the helper that emits that IndexOf call was passed the wrong node from the pattern: it was being passed the object representing the (?:.|\n) any-set rather than the “*/” literal, which resulted in it emitting the equivalent of IndexOfAnyInRange((char)0, ‘\uFFFF’) rather than the equivalent of IndexOf(“*/”). Oops. It was still functionally correct, in that the IndexOfAnyInRange call would successfully match the first character and the loop would re-evaluate from that location, but that means that rather than efficiently skipping using SIMD over a bunch of positions that couldn’t possibly match, we were doing non-trivial work for each and every position along the way.”

As in the IO section above, some of the optimizations come in the form of analyzers that recommend an optimization that the user can apply rather than something that the runtime can do automatically.

“[…] the .NET 10 SDK includes a new analyzer related to Regex. It’s oddly common to see code that determines whether an input matches a Regex written like this: Regex.Match(…).Success. While functionally correct, that’s much more expensive than Regex.IsMatch(…). For all of the engines, Regex.Match(…) requires allocating a new Match object and supporting data structures (except when there isn’t a match found, in which case it’s able to use an empty singleton); in contrast, IsMatch doesn’t need to allocate such an instance because it doesn’t need to return such an instance (as an implementation detail, it may still use a Match object, but it can reuse one rather than creating a new one each time).”

MemoryExtensions

“These overloads all parallel existing methods, but remove the IEquatable<T> (or IComparable<T>) constraint on the generic method parameter and accept an optional IEqualityComparer<T>? (or IComparer<T>). When no comparer or a default comparer is supplied, they can fall back to using the same vectorized logic for relevant types, and otherwise can provide as optimal an implementation as they can muster, based on the nature of T and the supplied comparer.”

This part is very interesting because you see how the improvements to MemoryExtensions led to SearchValues being faster, which, in turn, led to methods like Normalize and Contains being faster (especially when working with strings that are automatically treated as Spans wherever possible).

JSON

A good method to know is RemoveAll(), which accepts a lambda to filter for the elements to remove. If, instead of looping over the items and calling RemoveAt(n), you write _arr.RemoveAll(static n => n!.GetValue<int>() % 2 == 0), you get a huge performance benefit because RemoveAll() adjusts the underlying buffer only once rather than on each call to remove each individual item.

“With JSON being used as an encoding for many modern protocols, streaming large JSON payloads has become very common. And for most use cases, it’s already possible to stream JSON well with System.Text.Json. However, in previous releases there wasn’t been a good way to stream partial string properties; string properties had to have their values written in one operation. If you’ve got small strings, that’s fine. If you’ve got really, really large strings, and those strings are lazily-produced in chunks, however, you ideally want the ability to write those chunks of the property as you have them, rather than needing to buffer up the value in its entirety. dotnet/runtime#101356 augmented Utf8JsonWriter with a WriteStringValueSegment method, which enables such partial writes. […] These modern protocols often transmit large blobs of binary data within the JSON payloads. Typically, these blobs end up being Base64 strings as properties on some JSON object. Today, outputting such blobs requires Base64-encoding the whole input and then writing the resulting bytes or chars in their entirety into the Utf8JsonWriter. To address that, dotnet/runtime#111041 adds a WriteBase64StringSegment method to Utf8JsonWriter.”

Cryptography

“A ton of effort went into cryptography in .NET 10, almost entirely focused on post‑quantum cryptography (PQC). PQC refers to a class of cryptographic algorithms designed to resist attacks from quantum computers, machines that could one day render classic cryptographic algorithms like Rivest–Shamir–Adleman (RSA) or Elliptic Curve Cryptography (ECC) insecure by efficiently solving problems such as integer factorization and discrete logarithms. With the looming threat of “harvest now, decrypt later” attacks (where a well-funded attacker idly captures encrypted internet traffic, expecting that they’ll be able to decrypt and read it later) and the multi-year process required to migrate critical infrastructure, the transition to quantum‑safe cryptographic standards has become an urgent priority. In this light, .NET 10 adds support for ML-DSA (a National Institute of Standards and Technology PQC digital signature algorithm), Composite ML-DSA (a draft Internet Engineering Task Force specification for creating signatures that combine ML-DSA with a classical crypto algorithm like RSA), SLH-DSA (another NIST PQC signature algorithm), and ML-KEM (a NIST PQC key encapsulation algorithm).”

Conclusion

Overall, this is another amazing document—a book—that is edited to an incredibly high quality. I didn’t notice any grammatical, formatting errors, or typos (maybe a missing `?` on IComparer<T> in “These overloads all parallel existing methods, but remove the IEquatable<T> (or IComparable<T>) constraint on the generic method parameter and accept an optional IEqualityComparer<T>? (or IComparer<T>).” or when he wrote “frequently-requested” (the hyphen is only correct with adjectives, not adverbs).

The article A Critical Look at MCP by Rasmus Holm (Raz Blog) discussing many of the drawbacks of MCP as it is currently conceived. One of them is the push to build everything in Python, which is a dynamic language that’s better-designed than JavaScript, but isn’t a lot better at helping users write maintainable code.

“Am I being pretentious/judgmental in thinking that people in AI only really know Python, and the “well, it works on my computer” approach is still considered acceptable? This should be glaringly obvious to anyone that ever tried to run anything from Hugging Face.

“If you want to run MCP locally, wouldn’t you prefer a portable language like Rust, Go, or even VM-based options such as Java or C#?”

Captain Jean-Luc PicardI’ve been having discussions with people at work about MCP. This post made me think that I haven’t been clear about my attitude toward it. I think it would be amazing if we could pose natural language queries to machines and have them do things for us. Absolutely. Who doesn’t want to just order “Tea. Earl grey. Hot.”?

My doubts are more specific to MCP itself, technically, as a protocol. This article is highly technical, but it boils down to: MCP is such a hype-y protocol right now and it’s so technically shaky that we have a responsibility to not just grab the first damned thing that shows up and make it the standard.

We did that with JavaScript and it took 2 years until it was everywhere and over 20 years until it was an actual professional tool. I’m an old man and, looking back, very often our industry is just stepping on rakes that are right there.

I just to clarify that I’m pushing back on the implementation not the idea.

This is a nice explanation of how CSS is a declarative language, where you describe the metadata of your styles. The layout algorithm determines which property values affect the size and position of the element. Generally the properties position and display properties determine which layout algorithm is used for a given element. The layouts are,

CSS makes sense when you realize it's a collection of algorithms by Kevin Powell (YouTube)

• Normal flow layout (MDN) (selected by default)
• Inline layout (MDN) (selected by default for inline elements)
• Flexible box layout (MDN) (selected with display: flex)
• Grid layout (MDN) (selected with display: grid)
• Inline layout (MDN) (selected with display: grid)
• Multi-column layout (MDN) (selected with display: grid)
• Positioned layout (MDN) (selected with display: grid)
• Flow layout (MDN) (selected float: left or float: right)

Most properties work the same in all layouts. Some properties only have an effect in a specific layout mode, e.g., grid-template-columns is ignored if the layout is not grid. Other properties are interpreted differently or completely ignored depending on layout mode, e.g., width and margin are ignored in the inline layout.

This is a nice ~13-minute explanation of how CSS is a declarative language, where you describe the metadata of your styles. The layout algorithm determines which property values affect the size and position of the element.

CSS makes sense when you realize it's a collection of algorithms by Kevin Powell (YouTube)

Generally the properties position and display properties determine which layout algorithm is used for a given element. The layouts are,

Kevin Powell

• Normal flow layout (MDN) (selected by default)
• Inline layout (MDN) (selected by default for inline elements)
• Flexible box layout (MDN) (selected with display: flex)
• Grid layout (MDN) (selected with display: grid)
• Inline layout (MDN) (selected with display: grid)
• Multi-column layout (MDN) (selected with display: grid)
• Positioned layout (MDN) (selected with display: grid)
• Flow layout (MDN) (selected float: left or float: right)

Most properties work the same in all layouts. Some properties only have an effect in a specific layout mode, e.g., grid-template-columns is ignored if the layout is not grid. Other properties are interpreted differently or completely ignored depending on layout mode, e.g., width and margin are ignored in the inline layout.

I just saw a neat code example from a Dutch government project (function starting at line 182). The commentator who posted it at Reddit wrote,

“Some people laughed at it and suggested all kind of clever one liners to replace it, but to me, that if statement is perfect. The intent is immediately clear and bugs are easy to spot. This is the kind of code you want in critical apps.”

Here’s the code.

private static string GetPercentageRounds(double percentage)
{
    if (percentage == 0)
        return "⚪⚪⚪⚪⚪⚪⚪⚪⚪⚪";
    if (percentage > 0.0 && percentage <= 0.1)
        return "🔵⚪⚪⚪⚪⚪⚪⚪⚪⚪";
    if (percentage > 0.1 && percentage <= 0.2)
        return "🔵🔵⚪⚪⚪⚪⚪⚪⚪⚪";
    if (percentage > 0.2 && percentage <= 0.3)
        return "🔵🔵🔵⚪⚪⚪⚪⚪⚪⚪";
    if (percentage > 0.3 && percentage <= 0.4)
        return "🔵🔵🔵🔵⚪⚪⚪⚪⚪⚪";
    if (percentage > 0.4 && percentage <= 0.5)
        return "🔵🔵🔵🔵🔵⚪⚪⚪⚪⚪";
    if (percentage > 0.5 && percentage <= 0.6)
        return "🔵🔵🔵🔵🔵🔵⚪⚪⚪⚪";
    if (percentage > 0.6 && percentage <= 0.7)
        return "🔵🔵🔵🔵🔵🔵🔵⚪⚪⚪";
    if (percentage > 0.7 && percentage <= 0.8)
        return "🔵🔵🔵🔵🔵🔵🔵🔵⚪⚪";
    if (percentage > 0.8 && percentage <= 0.9)
        return "🔵🔵🔵🔵🔵🔵🔵🔵🔵⚪";

    return "🔵🔵🔵🔵🔵🔵🔵🔵🔵🔵";
}

This is a cool example because it demonstrates how easy it is to understand the return value when you use a separate constant for each “progress bar” increment instead of using something like new string('🔵', 5), which, as we’ll see below, doesn’t even compile.

Still, all but the first condition needlessly checks the lower-bound already guaranteed by the previous step. At the very least, we can reduce it to the following:

private static string GetPercentageRounds(double percentage)
{
    if (percentage == 0)
        return "⚪⚪⚪⚪⚪⚪⚪⚪⚪⚪";
    if (percentage <= 0.1)
        return "🔵⚪⚪⚪⚪⚪⚪⚪⚪⚪";
    if (percentage <= 0.2)
        return "🔵🔵⚪⚪⚪⚪⚪⚪⚪⚪";
    if (percentage <= 0.3)
        return "🔵🔵🔵⚪⚪⚪⚪⚪⚪⚪";
    if (percentage <= 0.4)
        return "🔵🔵🔵🔵⚪⚪⚪⚪⚪⚪";
    if (percentage <= 0.5)
        return "🔵🔵🔵🔵🔵⚪⚪⚪⚪⚪";
    if (percentage <= 0.6)
        return "🔵🔵🔵🔵🔵🔵⚪⚪⚪⚪";
    if (percentage <= 0.7)
        return "🔵🔵🔵🔵🔵🔵🔵⚪⚪⚪";
    if (percentage <= 0.8)
        return "🔵🔵🔵🔵🔵🔵🔵🔵⚪⚪";
    if (percentage <= 0.9)
        return "🔵🔵🔵🔵🔵🔵🔵🔵🔵⚪";

    return "🔵🔵🔵🔵🔵🔵🔵🔵🔵🔵";
}

I would elect to go further, preserving the clarity in constants (or maybe a comment) to avoid repetition in the code.

First, let’s write a few tests with NUnit.

[TestCase(0.00, "⚪⚪⚪⚪⚪⚪⚪⚪⚪⚪")]
[TestCase(0.10, "🔵⚪⚪⚪⚪⚪⚪⚪⚪⚪")]
[TestCase(0.11, "🔵⚪⚪⚪⚪⚪⚪⚪⚪⚪")]
[TestCase(0.19, "🔵⚪⚪⚪⚪⚪⚪⚪⚪⚪")]
[TestCase(0.20, "🔵🔵⚪⚪⚪⚪⚪⚪⚪⚪")]
[TestCase(0.30, "🔵🔵🔵⚪⚪⚪⚪⚪⚪⚪")]
[TestCase(0.40, "🔵🔵🔵🔵⚪⚪⚪⚪⚪⚪")]
[TestCase(0.50, "🔵🔵🔵🔵🔵⚪⚪⚪⚪⚪")]
[TestCase(0.60, "🔵🔵🔵🔵🔵🔵⚪⚪⚪⚪")]
[TestCase(0.70, "🔵🔵🔵🔵🔵🔵🔵⚪⚪⚪")]
[TestCase(0.80, "🔵🔵🔵🔵🔵🔵🔵🔵⚪⚪")]
[TestCase(0.90, "🔵🔵🔵🔵🔵🔵🔵🔵🔵⚪")]
[TestCase(1.00, "🔵🔵🔵🔵🔵🔵🔵🔵🔵🔵")]
public void TestBubbles(double percentage, string expectedOutput)
{
    var actualOutput = GetPercentageRounds(percentage);

    Assert.That(actualOutput, Is.EqualTo(expectedOutput));
}

First, let’s just reflect that now it’s the tests that document how the API works! That means that now we don’t have to lean so hard on the examples being obvious in the code itself. We can optimize the code for maintainability because the test offers the high-level explanation. When I refactor this code, I’m not removing the in-code documentation—I’ve simply moved it to the test.

Next, let’s give in to our refactoring instincts and see if a shorter formulation of the algorithm is also understandable. The algorithm is now:

1. Build constant buffers for empty and filled.
2. Calculate the portion of each of these buffers to include in the result (filledCount and emptyCount).
3. Copy the correct number of characters from the buffers using the C# range-operator.

private static string GetPercentageRounds(double percentage)
{
    const string empty = "⚪⚪⚪⚪⚪⚪⚪⚪⚪⚪";
    const string filled = "🔵🔵🔵🔵🔵🔵🔵🔵🔵🔵";

    var filledCount = (int)Math.Floor(percentage * 10);
    var emptyCount = 10 - filledCount;

    return filled[..filledCount] + empty[..emptyCount];
}

This doesn’t work, though!

The tests fail. For example, the test for 0.8 returns “🔵🔵🔵🔵⚪⚪” instead of “🔵🔵🔵🔵🔵🔵🔵🔵⚪⚪”. What’s going on?

We would get another hint as to what is going on if we were to refactor the constant declarations to use each symbol only once. For example, I could create the string with a special constructor, as shown below.

var empty = new string ('⚪', 10);
var filled = new string ('🔵', 10);

This avoids repeating the symbol several times but it’s probably also not as clear what’s happening. It also no longer uses constants—initialized once and stored in the app—so we’re allocating new strings each time. We could declare them as static instance variables so that they are allocated only once. However, that would also mean that we don’t declare them locally in the method, which again decreases readability.

On top of that, though, the second initialization doesn’t even compile!

🔵 is not a single character

Strings are encoded in UTF-16 (the standard for .NET). [1] In this encoding, the “⚪” is represented with one byte, while “🔵” is represented with two bytes. That knowledge, together with knowing that the range operator works with bytes, explains why we only got half as many filled-in symbols as expected.

Knowing this, we can revert to the original constants and fix the algorithm as follows (code-change is highlighted).

private static string GetPercentageRounds(double percentage)
{
    const string empty = "⚪⚪⚪⚪⚪⚪⚪⚪⚪⚪";
    const string filled = "🔵🔵🔵🔵🔵🔵🔵🔵🔵🔵";

    var filledCount = (int)Math.Floor(percentage * 10);
    var emptyCount = 10 - filledCount;

    return filled[..(2 * filledCount)] + empty[..emptyCount];
}

OK. Now it’s working. We now have two questions:

1. Can we avoid the “hack” for UTF-16 in our calculation?
2. The code is now more maintainable; is the code still as understandable as before?

Let’s tackle the first one. It turns out that there is a standard way of indexing by grapheme but you have to opt in to it by using a StringInfo object, which offers a method named SubstringByTextElements().

private static string GetPercentageRounds(double percentage)
{
    const string empty = "⚪⚪⚪⚪⚪⚪⚪⚪⚪⚪";
    const string filled = "🔵🔵🔵🔵🔵🔵🔵🔵🔵🔵";

    var filledCount = (int)Math.Floor(percentage * 10);
    var emptyCount = 10 - filledCount;

    return new StringInfo(filled).SubstringByTextElements(0, filledCount) + new StringInfo(empty).SubstringByTextElements(0, emptyCount);
}

Now our code is no longer making assumptions about how many bytes represent our empty and filled symbols. But is it better? No. It is absolutely less legible than even the previous version. It also allocates two new StringInfo objects ever time it executes.

Is it even necessary? Also no.

Why wouldn’t it be necessary? In the general case, we have to stay flexible and make sure that we’re extracting the correct number of graphemes (not characters), but we don’t have a general case here. We have two constant strings in a known encoding. We know that we can index by byte into the empty string and we know that we can index by two bytes into the filled string. These are constants. They will not change. We can make assumptions based on that.

That means, after this little excursion, that we’ll use our original version but we will also no longer consider it a hack.

This takes us to the final point: is the new version more legible than the original? I think that it is. At first blush, the original looks like it’s very self-explanatory—you can see how the progress bar is built—but you also have many more points of logic to check to verify that it’s actually working as expected. While you can use the test I’ve defined above to check all of the logic, there are many more conditions to check when something goes wrong. We measure the number of paths through a piece of logic as cyclomatic complexity (Wikipedia). The lower the better.

We have learned that, when you program in the original way, you may actually save time! The original formulation didn’t have to concern itself with encodings because it wasn’t slicing strings. The original programmer didn’t even need to be aware that some characters are encoded with multiple bytes whereas others are encoded with a single byte. They didn’t even have to know what a byte was at all!

Food for thought.

As almost always, there isn’t a “best solution” for all situations. There is a solution that minimizes drawbacks for the given requirements, but not for all possible requirements. If one of the requirements were: the reader need not know what a byte is, then the original solution would be more appropriate.

The final version below has lower cyclomatic complexity, uses constants to indicate what the result will actually look like, and explains its algorithm reasonably well, if you understand percentages. I’ve included a comment to explain why we double the number of bytes to select from filled.

private static string GetPercentageRounds(double percentage)
{
    const string empty = "⚪⚪⚪⚪⚪⚪⚪⚪⚪⚪";
    const string filled = "🔵🔵🔵🔵🔵🔵🔵🔵🔵🔵";

    var filledCount = (int)Math.Floor(percentage * 10);
    var emptyCount = 10 - filledCount;

    // Each 🔵 is two bytes in UTF-16
    return filled[..(2 * filledCount)] + empty[..emptyCount];
}

The article Function calling using LLMs by Kiran Prakash writes,

“It’s important to emphasize that when using function calling, the LLM itself does not execute the function. Instead, it identifies the appropriate function, gathers all required parameters, and provides the information in a structured JSON format. This JSON output can then be easily deserialized into a function call in Python (or any other programming language) and executed within the program’s runtime environment.”

This is an approach that works very well when you don’t have a testing environment: build a plan, evaluate validity of the plan, and then apply the plan after verification. You should also be able to slice the work into sub-tasks to make verification more reliable. This is the approach I took for a PowerShell script that runs against an ADOS instance: it’s production data, so you really want to be sure what is going to be executed.

In the implementation, you can see how the code he writes prepares the query to the LLM in a structured way with the required context in an attempt to guide the result. Happily, he begins by writing unit tests!

Function-calling with an LLMThis is another good step-by-step example of working with an LLM, but for a different task: it’s using an LLM as an interpreter for the user’s input. It’s basically a way of adding a natural-language “search-like” interface to an app without forcing the user to structure their input, without developing an UI, and without writing a parser. The advantage is that you get a way of querying a potentially large API surface in a way that in more amenable to more users.

I think of an example from Markus Schenkel from Cudos, who talked about using an MCP plugin for working with a CAD/CAM program—apps that notoriously have dozens of toolbars and thousands of functions. He could formulate his “novice” request as text, and the LLM, together with the mapping to tool functionality, made relatively good guesses about what he was trying to do. It often took a few attempts—but he was able to accomplish his task, whereas he would have either given up or had to invest a lot more time to get it done otherwise.

I think this is great for products that are in proof-of-concept stage, so that you don’t iterate on UIs too early in the design process. But we also have to be aware that we have UIs for a reason. Once there’s a well-established set of use cases and functionality, then it’s unclear that making users continue to use a command-line interface where they compose text is better than a GUI.

At any rate, the article is filled with detail and code (in Python) for using an LLM in the way described above. There’s a section on refactoring at the end, a comparison to the rules-engine-based approach that this technique seeks to replace, and also a comparison of function-calling with MCP.
 

The article The null check that didn’t check for nulls by Oren Eini (Ayende) points out an interesting and subtle difference in code-generation, depending on whether you use the var keyword. Using var in pattern-matching might lead to a pattern that looks like it checks for null but doesn’t. You can see and play with a live example (SharpLab.IO) but I’ve replicated the examples below.

This is the problematic example:

string Test1(List<string> strs)
{
    if(strs is [var s])
    {
        return s;
    }
    return string.Join(",", strs);
}

It’s basically saying that the pattern should match anything that’s a collection with one element. Since the type is obvious from the method signature’s parameter strs, we use var instead of string. That generates the following code.

internal static string <Main>$>g__Test1|0_0(List<string> strs)
{
    if (strs != null && strs.Count == 1)
    {
        return strs[0];
    }
    return string.Join(",", strs);
}

Note that it returns the first element without checking it for null.

If you change the var to string, which, as noted above, is redundant, then the generated code includes a null-check.

string Test2(List<string> strs)
{
    if(strs is [string s])
    {
        return s;
    }
    return string.Join(",", strs);
}

This is the generated code for the example above.

internal static string <Main>$>g__Test2|0_1(List<string> strs)
{
    if (strs != null && strs.Count == 1)
    {
        string text = strs[0];
        if (text != null)
        {
            return text;
        }
    }
    return string.Join(",", strs);
}

If you instead use { } to indicate that you want to match a non-null object, then you also get the null-check.

string Test3(List<string> strs)
{
    if(strs is [{} s])
    {
        return s;
    }
    return string.Join(",", strs);
}

This is the generated code for the example above. It is the same as the second example that uses string for the matched parameter.

internal static string <Main>$>g__Test3|0_2(List<string> strs)
{
    if (strs != null && strs.Count == 1)
    {
        string text = strs[0];
        if (text != null)
        {
            return text;
        }
    }
    return string.Join(",", strs);
}

The article Zed: The Fastest AI Code Editor by Richard Feldman (Zed) includes a great description and video that shows off the behavior of their new “agent” feature.

“The entire Zed code editor is open source under GPL version 3, and scratch-built in Rust all the way down to handcrafted GPU shaders and OS graphics API calls. Zed’s new AI capabilities are also open-source, just like the rest of the editor, so you can see exactly what the new Agent Panel is doing under the hood.”

This editor is very, very smooth and more powerful than a standard Visual Studio Code. It’s also so much faster. However … it’s currently MacOS and Linux-only. The Windows version is in an early-access phase.

'Agent finished' notificationEven if you can’t use the editor, the ~5-minute video at the beginning of the post is absolutely what I’ve been looking for: how do you use these tools for real. The video demonstrates the following:

• Using the inline-completion to make small edits
• Using the chat-AI agent (the tool has access to many actions in the editor).
• Running a larger request/action against a large code base (they use the code of Zed itself, written in Rust).

  • The request is to make the number of most recently used values in a list configurable via settings.
  • The settings object already exists.
  • The list already exists.
  • It’s hard-coded to six elements right now.
• Viewing the steps taken in running the request.

• Reviewing and adjusting the proposed changes.

  “The diff is fully editable, so you can easily make changes to whatever the model came up with. It supports multicursor editing, language server integrations, and all the speed you love from the rest of Zed.”

• Noting that one of the proposed changes is something that even a senior developer might have forgotten to do in a first attempt at the feature (updating settings
• Final review in a Git diff.

This is hands-down the best demonstration I’ve seen of extending a workflow comprising what the author nicely describes as deterministic tools—I’ve been calling them analysis-based tools—with AI-based tools (and agents). The section on costs is remarkably fair and open.
 

A while back, I wrote Stop trying so hard to use pattern-matching. I stand by everything i wrote there. I was recently mentoring a very clever programmer who’s new to C# but has cut his teeth on Rust. We were discussing switch statements vs. switch expressions. Which pattern-matching features can you use where? Which features can you combine?

The article Tutorial: Use pattern matching to build type-driven and data-driven algorithms (Learn Microsoft) offers a good introduction. Pattern-matching on objects is lovely (and its been available since C# 7.0 [1] (2017)). The version they were using still used “switch statements”. There’s another level called “switch expressions” (available since C# 9 (2020)) that they could have used if they were returning a value. The article C# 9.0: Pattern Matching in Switch Expressions by Thomas Claudius Huber provides the following example,

string favoriteTask = obj switch
{
  Developer dev when dev.YearOfBirth == 1980 => $"{dev.FirstName} listens to metal",
  Developer dev => $"{dev.FirstName} writes code",
  Manager _ => "Create meetings",
  _ => "Do what objects do",
};

Speaking of syntactic sugar, you can check out what the compiler would generate using the SharpLab.IO. Throw in any compiling code on the left, and you get the “lowered” version on the right.

If you throw in the example from above with a bit of extra code to make it compile,

using System;

public class C
{
  public void M(object obj)
  {
    string favoriteTask = obj switch
    {
      Developer { YearOfBirth: >= 1980 and <= 1989 and not 1984 } dev
        => $"{dev.FirstName} listens to heavy metal while coding",
      Developer dev => $"{dev.FirstName} writes code",
      Manager _ => "Create meetings",
      _ => "Do what objects do",
    };

  }

  private class Developer
  {
    public int YearOfBirth { get; }
    public string FirstName { get; } = string.Empty;
  }

  private class Manager { }
}

You can see that the generated logic is quite straightforward. The snippet below elides the generated code for the Developer and Manager classes. It’s not how I would have written it manually, but I bet it’s pretty efficient.

[NullableContext(1)]
public void M(object obj)
{
    Developer developer = obj as Developer;
    string text;
    if (developer == null)
    {
        text = ((!(obj is Manager)) ? "Do what objects do" : "Create meetings");
    }
    else
    {
        int yearOfBirth = developer.YearOfBirth;
        if (yearOfBirth >= 1980 && yearOfBirth <= 1989 && yearOfBirth != 1984)
        {
            Developer developer2 = developer;
            text = string.Concat(developer2.FirstName, " listens to heavy metal while coding");
        }
        else
        {
            text = string.Concat(developer.FirstName, " writes code");
        }
    }
    string text2 = text;
}

I recently had a conversation about the pros and cons of using Angular and I found this year-old article that I’d prepared from my notes but never published. The article
Two-way binding between Signals and Query Params by Julio Castro (Software Engineering Corner by Zühlke Engineers) includes the following code snippet.

@Component({
  selector: "app-root",
  standalone: true,
  imports: [AsyncPipe],
  template: `
    <h1>Signals Demo</h1>
    <p>Your first name is: {{ firstName$ | async }}</p>
  `,
})
export class AppComponent {
  private activatedRoute = inject(ActivatedRoute);

  firstName$ = this.activatedRoute.queryParams.pipe(
    map((allQueryParams) => allQueryParams["firstName"])
  );
}

I cannot describe how gross I think Angular code is. None of this is “using the platform”. This is all JavaScript-first. It’s all custom, untyped, string-matching, gobbledygook. The firstName$ in the template isn’t checked. There are no type-safe views. [1] What the hell does | async do? I’m sure it’s convenient, but this is more obtuse-looking than modern React.

The injection is also just magic that you have to know about. And why is it injected differently than the AsyncPipe? There are probably good reasons for it, in Angular, but it looks pretty slapdash and ad=hoc as an API. It’s like there’s a different symbol or character or concept for every possible thing. The imports is in a custom place. It’s all packed into a @Component decorator that does a bunch of magic for you to build what is probably a web component (but I’m not sure). They wrapped every single possible API in something custom to Angular.

I hope I’m wrong, but this is so unappealing.

Reading a bit further and we see an example where some of the noise—e.g., the async pipe—has been removed because of the magic of signals.

If you’re learning Angular, you’re not learning anything portable about web-programming. You won’t know HTML, you won’t know CSS, you won’t even necessarily know JavaScript or the browser APIs. You don’t use the platform. It’s a shame because the platform is already so powerful. In the old days, you needed a framework to shield you from the differences. Nowadays, the platform is more than well-specified, -supported, and -implemented to just write to directly. Learning the platform API is just as easy as learning whatever I’m seeing in Angular.

In fairness to the article, though, it’s well-written and offers some good techniques for making the best of a bad situation if you have to work in Angular. 🙃

But then there’s this.

“Since we are accessing the value of the allQueryParams signal in the effect, it will run every time this signal gets updated, which happens every time Angular emits a new value in the activatedRoute.queryParams observable.

“Inside the effect, we are just updating the value of our queryParamValue signal. For that, notice that we need to pass the allowSignalWrites: true option. This is necessary because updating signals in effects could lead to infinite loops and unexpected and intricate situations in general.”

This is the same kind of black magic for real-life situations as you see in React these days. I’m still a fan of using MobX for the state model, then attaching it to pure reactive web components. I’m still deciding whether that will scale to what I need, but I’m more and more convinced that none of the huge frameworks are the way to go. They’re just so much wrapping and bizarre APIs that feel legacy before they’re even officially released.

I’m not going to copy it in here but the final version of the read/write signal service based on query-parameter values is 41 lines of hairball code. Do I know how much code it would be to achieve something similar outside of Angular? No. No, I don’t. I just know that if I ended up having to learn how to do it and write it—and even if it ended up being more code—I would have learned the general platform and built a service that can work in any web site, not just one framework.

This video is just under 30 minutes and provides a lot of useful tips about how to optimize web pages. It’s almost a year old, but a lot of the optimizations are good to know, even though they won’t apply to most pages out there. It’s good to know how the browser works and which heuristics it uses to determine what can be optimized. Knowing these things helps you avoid accidentally formulating your web pages in ways that slow things down unnecessarily. You’ll be less likely to suffer under load.

Optimizing INP: A deep dive by Chrome for Developers (YouTube)

The author goes through several optimizations.

• He starts by showing how to turn on the “mid-tier mobile” testing mode, which uses fast 3G and slows the CPU down 4x. This makes it easier to spot problems on a developer-class desktop/laptop.
• He then shows how to set up and use the profiler, zooming in and out of the extremely rich data recorded for every interaction.
• He discovers and removes a polyfill that’s no longer needed. It turns out that that version of polyfill was broken and always active—regardless of whether the feature was supported natively.
• Another fix was to remove the background blur when making an element inert (MDN) because it was engaging the GPU and causing a much longer paint when the browser had to animate the drop-shadow moving across the blurred element.
• Another fix involved simply moving an interaction away from the initial event handler by executing it in a timer instead. He used a dead-simple debouncing technique to ensure that only the most recent task would be executed.
• Another fix was to remove complex logic for avoiding setting the display property on a DOM element. The solution there is to simply let the browser do its thing; it’s much more optimized than you think. The code that was trying to avoid touching the DOM was much slower than actually setting a DOM property.
• Another fix was to defer and chunk appending results as well as setting styling for found terms by using async.

Extension syntax in C#14Since this feature is being touted for C# 14—this time it’s coming for real!—I thought it would be good to refresh what I’d already learned about it. The title is a bit hyperbolic but it’s quite an interesting feature. It’s basically protocol extension from Swift for C#. It’s .NET’s answer to extending extension methods to properties and, probably, operators. You can’t add state, as far as I can tell. But that isn’t so surprising.

The video below discuss the proposal as it looked for C#13. The pages What’s new in C# 14: Extension members and Extension declaration (C# Reference) offer more insight into the current shape of the feature. It seems like they mean it this time and that it will land in November 2025.

The Insane C# 13 Feature That Changes Everything by Nick Chapsas (YouTube)

What it primarily is, though, is further work on making it easier to transition APIs. We got the first batch of support with default interface implementations. This feature will allow to smooth migrations even more. They will also allow us to “add” properties to types that then introduce their own version of those properties in future versions but that’s OK, I think. It means that every added property will be a potential breaking change for someone but maybe it will make us start categorizing breaking changes.

There are implicit extensions, which are pretty much a new way of defining extension methods, but with support for proprties. The following example shows how the property IsLead will be available for any Person without modifying that type. This doesn’t seem much different than existing extension methods, other than support for properties, where the this keyword stands in for the parameter that would otherwise have been passed in a classic extension method.

public implicit extension PersonExtension for Person
{
    public bool IsLead
        => this.Organization
            .Teams
            .Any(team => team.Lead == this);
}

There are also explicit extensions, which are a way of specifying extensions to types that are neither implemented nor inherited, but are instead given to a type without coercion. That is, you can define a type that can be applied to another type (e.g., Lead for Person in the example below), which makes more methods and properties available. It’s kind of confusing without an example.

public explicit extension Lead for Person
{
    public IEnumerable<Team> Teams 
        => this.Organization
            .Teams
            .Where(team => team.Lead == this);
}

var person = new Person();
var personTeams = person.Teams; // Compile error
Lead lead = person;
var leadTeams = lead.Teams; // OK

While this might look like a cast, it’s not, because Person doesn’t implement Lead—it’s extended by Lead in code that isn’t necessarily associated with the code that defines Person.

In the latest syntax, the example above would look a bit different but the idea is the same. It looks like they’ve simplified it a bit.

public static class PersonExtension
{
    extension(Person source)
    {
        public bool IsLead
        => source.Organization
            .Teams
            .Any(team => team.Lead == this);;
    }
}

• There a talk called What’s new in C# 13 by Mads Torgersen and Dustin Campbell (Microsoft Build) for which you have to register. The video will probably come out later on YouTube, though.
• There’s the original [Proposal]: Extensions #5497 (GitHub)
• The most informative link (so far) is .NET Announcements and Updates from Microsoft Build 2024 by .NET Team (Microsoft Dev Blogs)

This is another 46-minute, 10-month-old video from the last Build conference that I found extremely helpful in explaining what .NET Aspire is and what it’s good for.

Demystify cloud-native development with .NET Aspire | BRK181 by Microsoft Developer (YouTube)

Damian Edwards and David Fowler do a soup-to-nuts demonstration of Aspire. It basically lets you configure your multi-project, distributed projects with code rather than with YAML (e.g. dockercompose.yml). Instead, it writes the files for you and handles the deployment to Docker. This lets you much more easily create and configure things like email servers (for registration workflows), queues, databases, etc. Some of the resources run in Docker containers, some run on Azure if you want. There is a dashboard with deep telemetry, with very nice graphs showing how each service participates in a given request.
 

This 46-minute presentation by Scott Hanselman and Stephen Toub is ten months old but is still worth watching. I note below that one of the more significant things Toub shows is not any sort of programming wizardry, but column-selection in a text editor. Half of the things that people use AI for can be solved with column-select and judicious copy/paste.

'Highly Technical Talk' with Hanselman and Toub | BRK194 by Microsoft Developer (YouTube)

Another fantastic “deep dive” with these two: this time they’re optimizing the Humanizer library on-the-fly, on-stage, during a session. This feels nearly completely improvised. Kudos to these two gen-Xers, doing an old-school presentation of just plain programming bravura with no frills.

At 38:20, Toub shows how to use column-select to make changes, which wows the audience. I guess it’s really not such a well-known feature, but it’s an incredible productivity booster. Toub uses the mouse to select when he could have just used the keyboard to select the lines with Alt + Shift + down or by selecting the space and starting double-quote, then Alt + Shift + . to select subsequent matches. After that, he used the mouse again to select the end of the lines, but he could have just left the lines selected from before and hit End to jump to the end of all the lines. It’s good that he showed it but, as in previous videos, he’s a bit more of a “mouser” than I am.
 

I couldn’t call Docker from the command line. I had installed Docker a long time ago, but had just restored from a Time Machine backup, so my system was new but the applications had been restored. That meant that Docker had recorded that the executables had been sym-linked to the right folder (/usr/local/bin) but those links were part of the old, dead system.

Long story short, go to the settings, as shown below. If you’re in the situation that I was in, in which the app was out of sync with the system, then you’ll have to change something to get the app to apply the settings again.

Docker Settings

You can try to simply switch from User to System, click Apply & Restart but that didn’t work for me. It never asked for my password or confirmation. However, I found the instructions in this answer to docker-compose command not available for mac (StackOverflow), which gave the tip to change other things on the page to force Docker to make the update.

“I fixed it by going to Docker app -> settings -> Advanced

“In advanced I had to check off:”

• Allow the default Docker socket to be used (requires password)
• Allow privileged port mapping (requires password)

“Click Apply & Restart

“And enable again options:”

• Allow the default Docker socket to be used (requires password)
• Allow privileged port mapping (requires password)

“Click Apply & Restart”

If that still doesn’t work, then check whether you actually have a /usr/local/bin folder. Docker fails silently when the folder doesn’t exist. If you don’t have it, use an administrator and sudo to create the folder and try the procedure above again.

That did the trick for me.

If you’d like to link the tools to your user’s path ($HOME/.docker/bin) instead, then follow the instructions in this answer to Setting PATH environment variable in OSX permanently (StackOverflow) that show you how to modify the paths for your system.

The articlebook Performance Improvements in .NET 9 by Stephen Toub (.NET Blog) was published about six months ago. It contains a tremendous amount of interesting information, which I’ve attempted to summarize below, following the document structure in the original.

Tier 0

“Another tier 0 boxing example is dotnet/runtime#90496. There’s a hot path method in the async/await machinery: AsyncTaskMethodBuilder<TResult>.AwaitUnsafeOnCompleted (see How Async/Await Really Works in C# for all the details). It’s really important that this method be optimized well, but it performs various type tests that can end up boxing in tier 0. In a previous release, that boxing was deemed too impactful to startup for async methods invoked early in an application’s lifetime, so [MethodImpl(MethodImplOptions.AggressiveOptimization)] was used to opt the method out of tiering, such that it gets optimized from the get-go. But that itself has downsides, because if it skips tiering up, it also skips dynamic PGO, and thus the optimized code isn’t as good as it possibly could be. So, this PR specifically addresses those type tests patterns that box, removing the boxing in tier 0, enabling removing that AggressiveOptimization from AwaitUnsafeOnCompleted, and thereby enabling better optimized code generation for it.”

Loops

“In .NET 8, as part of the work to improve dynamic PGO, a more powerful graph-based loop analyzer was added that was able to recognize many more loops. For .NET 9 with dotnet/runtime#95251, that analyzer was factored out so that it could be used for generalized loop reasoning. And then with PRs like dotnet/runtime#96756 for loop alignment, dotnet/runtime#96754 and dotnet/runtime#96553 for loop cloning, dotnet/runtime#96752 for loop unrolling, dotnet/runtime#96751 for loop canonicalization, and dotnet/runtime#96753 for loop hoisting, many of these loop-related optimizations have now been moved to the better scheme. All of that means that more loops get optimized.”

ARM SVE

“There are multiple ways such an ISA impacts .NET, and in particular the JIT. The JIT needs to be able to be able to work with the ISA, understand the associated registers and be able to do register allocation, be taught about encoding and emitting the instructions, and so on. The JIT needs to be taught when and where it’s appropriate to use these instructions, so that as part of compiling IL down to assembly, if operating on a machine that supports SVE, the JIT might be able to pick SVE instructions for use in the generated assembly. And the JIT needs to be taught how to represent this data, these vectors, to user code. All of that is a huge amount of work, especially when you consider that there are thousands of operations represented. What makes it even more work is hardware intrinsics.”

“Designing and enabling the SVE support is a monstrous, multi-year effort, and while the support is functional and folks are encouraged to take it for a spin, it’s not yet baked enough for us to be 100% confident the shape won’t need to evolve (for .NET 9, it’s also restricted to hardware with a vector width of 128 bits, but that restriction will be removed subsequently). Hence, [Experimental].”

AVX512

“So the values are 0 1 1 0 1 0 0 0, which we read as the binary 0b01101000, which is 0x68. That byte is used as a “control code” to the vpternlog instruction to encode which of the 256 possible truth tables that exist for any possible (deterministic) Boolean combination of those inputs is being chosen. This PR then teaches the JIT how to analyze the tree structures produced by the JIT to recognize such sequences of Boolean operations, compute the control code, and substitute in the use of the better instruction. Of course, the JIT isn’t going to do the enumeration I did above; turns out there’s a more efficient way to compute the control code, performing the same sequence of operations but on specific byte values instead of Booleans.”

“This is beneficial for a variety of reasons, including less data to store, less data to load, and if the register containing this state needed to be spilled (meaning something else needs to be put into the register, so the value currently in the register is temporarily stored in memory), reloading it is similarly cheaper.”

All the considerations are mind-boggling. Does it fit in a cache line? How many registers does it use? Is it colocated with similar data? Is the data aligned on a boundary?

Vectorization

“Of course, you may then wonder, why wasn’t bool.TryFormat reverted to use the simpler code? The unfortunate answer is that this optimization only currently applies to array targets rather than span targets. That’s because there are alignment requirements for performing these kinds of writes, and whereas the JIT can make certain assumptions about the alignment of arrays, it can’t make those same assumptions about spans, which can represent slices of something else at unaligned boundaries. This is now one of the few cases where arrays are better than spans; typically span is as good or better. But I’m hopeful it will be improved in the future.”

Object Stack Allocation

“The hardest part of stack allocating objects is ensuring that it’s safe. If a reference to the object were to escape and end up being stored somewhere that outlived the stack frame containing the stack-allocated object, that would be very bad; when the method returned, those outstanding references would be pointing to garbage. So, the JIT needs to perform escape analysis to ensure that never happens, and doing that well is extremely challenging.”

This is the case where you have to be exceedingly clever in order to not have to let pessimism kill the feature entirely. That is, if you can’t prove enough, then you end up having to assume that escape is possible in too many cases—and the optimization ends up applying much less than you’d hoped it would.

VM

“The .NET runtime provides many services to managed code. There’s the GC, of course, and the JIT compiler, and then there’s a whole bunch of functionality around things like assembly and type loading, exception handling, configuration management, virtual dispatch, interop infrastructure, stub management, and so on. All of that functionality is generally referred to as being part of the coreclr virtual machine (VM).”

Mono

“We frequently say “the runtime,” but in reality there are currently multiple runtime implementations in .NET. “coreclr” is the runtime thus far referred to, which is the default runtime used on Windows, Linux, and macOS, and for services and desktop applications, but there’s also “mono,” which is mainly used when the form factor of the target application requires a small runtime: by default, it’s the runtime that’s used when building mobile apps for Android and iOS today, as well as the runtime used for Blazor WASM apps.”

“[…] when targeting WASM, the interpreter has a form of PGO where after methods have been invoked some number of times and are deemed important, it’ll generate WASM on-the-fly to optimize those methods. This tiering gets better in .NET 9 with dotnet/runtime#92981, which enables keeping track of which methods tiered up, and if the code is running in a browser, storing that information in the browser’s cache for subsequent runs. When the code then runs subsequently, it can incorporate the previous learnings to tier up better and more quickly.”

Threading / Debugger.NotifyOfCrossThreadDependency

“When you’re debugging a .NET process and you break in the debugger, it pauses all threads in the debuggee process so that nothing is making forward progress while you examine state. However, .NET debuggers, like the one in Visual Studio, support invoking properties and methods in the debuggee while debugging. That can be a big problem if the functionality being invoked relies on one of those paused threads to do something, e.g. if the property you access tries to take a lock that’s held by another thread or tries to Wait on a Task. To mitigate problems here, the Debugger.NotifyOfCrossThreadDependency method exists. Functionality that relies on another thread to do something can call NotifyOfCrossThreadDependency; if there’s no debugger attached, it’s a nop, but if there is a debugger attached, this signals the problem to the debugger, which can then react accordingly. The Visual Studio debugger reacts by stopping the evaluation but then by offering an opt-in option of “slipping” all threads, unpausing all threads until the evaluated operation completes, at which point all threads will be paused again, thereby again trying to mitigate any problems that might occur from the cross-thread dependency.”

VM

“The official .NET memory model has now been documented. However, some of the practices that were being employed in the core libraries (due to defensive coding or uncertainty of the memory model or out-of-date requirements) are no longer necessary. One of the main tools available for folks coding at a level where memory model is relevant is the volatile keyword / the Volatile class.”

“Marking fields or operations as volatile can come with an expense, depending on the circumstance and the target platform. For example, it can restrict the C# compiler and the JIT compiler from performing certain optimizations.”

Reflection

“Delegates in .NET are “multicast,” meaning a single delegate instance might actually represent multiple methods to be invoked; this is how .NET events are implemented. If I invoke a delegate, the delegate implementation handles invoking each constituent method, sequentially, in turn. But what if I want to customize the invocation logic? Maybe I want to wrap each individual method in a try/catch, or maybe I want to track the return values from all of the methods rather than just the last, or some such behavior. To achieve that, delegates expose a way to get an array of delegates, one for each method that’s part of the original.”

There are a lot of long chapters on number- and text-processing, which is fascinating but not eminently quotable. You can really see how so many of the various improvements build on each other to finally offer incredible speed improvements (e.g. Quaternion.Cosh()).

So many operations have been improved to reduce allocations to zero while reducing time to a few percent of the previous time, all often with even more code defined in C# rather in the JIT as native code (see Move memset/memcpy helpers to managed impl #98623 for an extreme example that touched 68 files in 48 commits). I find this to be quite elegant. It shows that the investment in the new C# constructs are paying off because it allows framework developers to build faster and better primitives without escaping to a different language and runtime. This, in turn, allows other skilled developers to benefit from the same. Not only that, but managed code is accessible to the GC whereas native code is not.

It’s very clear how .NET and C# are being positioned to take over numeric and text processing from Python and C++/C. Everything is being made more generic and funneled to vectorized types, which, in turn, map to the most optimal set of instructions for the myriad supported scenarios, like AOT, ARM, WASM, x64, x86, etc. It’s quite an incredible effort.

All of these things combine to make your regular expressions and text searches faster, even if you stick to the existing APIs. In some cases, there are new APIs to use, but not too many. Instead, the beauty of .NET 9 is that it will just make everything so much more efficient—faster and with fewer allocations and GC churn—without programmers having to do a thing. A true feat of engineering.

“[…] it’s important to recognize that many of the changes discussed thus far implicitly accrue to Regex. Regex already uses SearchValues, and so improvements to SearchValues benefit Regex (it’s one of my favorite things about working at the lowest levels of the stack: improvements there have a multiplicative effect, in that direct use of them improves, but so too does indirect use via intermediate components that instantly get better as the lower level does).”

DFA Limits

There is a ton of detail about the specifics of regular-expression optimization—enough to make your head spin. Like this:

“The non-backtracking implementation works by constructing a finite automata, which can be thought of as a graph, with the implementation walking around the graph as it consumes additional characters from the input and uses those to guide what node(s) it transitions to next. The graph is built out lazily, such that nodes are only added as those states are explored, and the nodes can be one of two kinds: DFA (deterministic) or NFA (non-deterministic). DFA nodes ensure that for any given character that comes next in the input, there’s only ever one possible node to which to transition. Not so for NFA, where at any point in time there’s a list of all the possible nodes the system could be in, and moving to the next state means examining each of the current states, finding all possible transitions out of each, and treating the union of all of those new positions as the next state. DFA is thus much cheaper than NFA in terms of the overheads involved in walking around the graph, and we want to fall back to NFA only when we absolutely have to, which is when the DFA graph would be too large: some patterns have the potential to create massive numbers of DFA nodes. Thus, there’s a threshold where once that number of constructed nodes in the graph is hit, new nodes are constructed as NFA rather than DFA. In .NET 8 and earlier, that limit was somewhat arbitrarily set at 10,000. For .NET 9 as part of this PR, analysis was done to show that a much higher limit was worth the memory trade-offs, and the limit was raised to 125,000, which means many more patterns can fully execute as DFA.”

“The inner matching loop is the hot path for a matching operation: read the next character, look up its minterm, follow the corresponding edge to the next node in the graph, rinse and repeat. Performance of the engine is tied to efficiency of this loop. These PRs recognized that there were some checks being performed in that inner loop which were only relevant to a minority of patterns. For the majority, the code could be specialized such that those checks wouldn’t be needed in the hot path.”

Span, Span, and more Span

“The introduction of Span<T> and ReadOnlySpan<T> back in .NET Core 2.1 have revolutionized how we write .NET code (especially in the core libraries) and what APIs we expose (see A Complete .NET Developer’s Guide to Span if you’re interested in a deeper dive.) .NET 9 has continued the trend of doubling-down on spans as a great way to both implicitly provide performance boosts and also expose APIs that enables developers to do more for performance in their own code.”

“One of the really nice optimizations the C# compiler added several years back was the ability to recognize when a new byte/sbyte/bool array was being constructed, filled with only constants, and directly assigned to a ReadOnlySpan<T>. In such a case, it would recognize that the data was all blittable and could never be modified, so rather than allocating an array and wrapping a span around it, it would blit the data into the assembly and then just construct a span around a pointer into the assembly data with the appropriate length.”

This is a wonderful optimization. Clever in a way that only a systems programmer would invent.

foreach (Range r in clientSecWebSocketProtocol.AsSpan().Split(','))
{
    if (clientSecWebSocketProtocol.AsSpan(r).Trim().Equals(acceptProtocol, StringComparison.OrdinalIgnoreCase))
    {
        return true;
    }
}

“In doing so, it becomes allocation-free, as this Split doesn’t need to allocate a string[] to hold results and doesn’t need to allocate a string for each segment: instead, it’s returning a ref struct enumerator that yields a Range representing each segment. The caller can then use that Range to slice the input. It’s yielding a Range rather than, say, a ReadOnlySpan<T>, to enable the splitting to be used with original sources other than spans and be able to get the segments in the original form.”

There is such a strong focus on structs and refs to make allocation-free code. And now we see how they leverage the recently introduced Range to provide indexes into a sequence that the calling code can decide how to extract. This offers maximum flexibility to the caller, as the algorithm isn’t making any costly decisions for it.

In this case, he’s discussing how they’ve made it relatively easy and intuitive to write code that searches a string without any allocations. The sequence doesn’t allocate, examining the chunk as a span doesn’t allocate, even the Trim() on a Span doesn’t allocate anything.

LINQ

There is a long chapter on LINQ optimizations that boils down to having cleaned up a ton of internal implementation to consolidate on a common base class for customer iteration-combinations like Where/First, Where/OrderBy, etc. Instead of testing for interfaces, it can now test for a single base class and perform a virtual rather than an interface dispatch (which is cheaper). This massive cleanup has the dual benefit of having made many, many LINQ operations 10, 20, and even 100 times faster—and many of them (if not most) are now completely allocation-free. Reducing allocations reduces churn in the GC, which also makes the app faster.

Core Collections

There is also a long chapter on dictionary optimizations. In particular, you can now store data in a dictionary with string keys but request an alternate view on the dictionary that lets you work with it as if it used ReadOnlySpan<char>, which can drastically reduce allocations as the spans you have don’t need to be converted to strings simply in order to do the lookups and stores. The changes apply to HashSets as well.

Compression

This is less about compression and more about the general philosophy and tactics underlying performance optimization in .NET (and, presumably, any runtime).

“It’s an important goal of the core .NET libraries to be as platform-agnostic as possible. Things should generally behave the same way regardless of which operating system or which hardware is being used, excepting things that really are operating system or hardware specific (e.g. we purposefully don’t try to paper over casing differences of different file systems). To that end, we generally implement as much as possible in C#, deferring down to the operating system and native platform libraries only when necessary.”

Networking

“dotnet/runtime#99364 changes the synchronization mechanism from using a pure lock-based scheme to a more opportunistic concurrency scheme that employs a first-layer of lockless synchronization. There’s now still a lock, but for the hot path it’s avoided as long as there are connections in the pool by using a ConcurrentStack<T>, such that renting is a TryPop and returning is a Push. ConcurrentStack<T> itself uses a lock-free algorithm, that’s a lot more scalable than a lock.”

“UrlEncode had a complicated scheme where it would UTF8-encode into a newly-allocated byte[], percent-encode in place in that (thanks to the ability to reinterpret cast with spans), and then use the resulting chars to create a new string. Instead, string.Create can be used, with all of the work done in-place in the buffer generated for that operation.”

“[…] updated UrlEncodeToBytes, using stack space instead of allocation for smaller inputs, and using SearchValues<byte> to optimize the search for invalid bytes.”

You can really see how the changes made over the last several versions allow a literal horde of open-source programmers to optimize the hell out of hot paths in the .NET library. Use Spans and ReadOnlySpans and ref structs and readonly ref structs to avoid allocations, allocate on the stack wherever you can when you can’t avoid allocations, return enumerators instead of allocating array results, use highly optimized building blocks like SearchValues and ConcurrentStack, which employ lock-free algorithms or include custom implementations for common patterns. It all adds up to being able to just write performant code by default, writing in a legible, maintainable, and concise high-level API that is carefully marshaled down to the processor by the compiler and/or the JIT to super-efficient IL and assembler code. You can visualize your code being analyzed and then sorted like Plinko chips until it finally lands in the processor cache as instructions.

Profiling with Benchmark.Net

“There’s another very handy nuget package, Microsoft.VisualStudio.DiagnosticsHub.BenchmarkDotNetDiagnosers, which contains additional “diagnosers” for BenchmarkDotNet. Diagnosers are one of the main extensibility points within BenchmarkDotNet, enabling developers to perform additional tracking and analyses over benchmarks. You’ve already seen me use some, including the built-in [MemoryDiagnoser(false)] and [DisassemblyDiagnoser]; there are other built-in ones we haven’t used in this post but that are helpful in various situations, like [ThreadingDiagnoser] and [ExceptionDiagnoser], but diagnosers can come from anywhere, and the aforementioned nuget package provides several more. The purpose of those diagnosers is to collect and export performance traces that Visual Studio’s performance tools can then consume. In my case, I want to collect a CPU trace, so as to understand where CPU consumption is going, so I added a [CPUUsageDiagnoser] attribute to my Tests class”

Lock-free programming

What does lock-free programming look like? You replace a lock with an atomic compare/exchange operation, usually in a loop (that’s why they’re sometimes called “spin locks”).

lock (this)
{
    _delta += value;
}

“it used an interlocked operation to perform the addition atomically. Here _delta is a double, and there’s no Interlocked.Add that works with double values, so instead the standard approach of using a loop around an Interlocked.CompareExchange was employed.”

double currentValue;
do
{
    currentValue = _delta;
}
while (Interlocked.CompareExchange(ref _delta, currentValue + value, currentValue) != currentValue);

Cache lines

Finally, an optimization that takes CPU cache lines into account. I hadn’t seen anything else that low-level so far.

“In this benchmark, one thread is incrementing _values[0] and the other thread is incrementing either _values[1] or _values[31]. That index is the only difference, yet the one accessing _values[31] is several times faster than the one accessing _values[1]. That’s because there’s contention here even if it’s not obvious in the code. The contention comes from the fact that the hardware works with memory in groups of bytes called a “cache line.” Most hardware has caches lines of 64 bytes. In order to update a particular memory location, the hardware will acquire the whole cache line. If another core wants to update that same cache line, it’ll need to acquire it. That back and forth results in a lot of overhead. It doesn’t matter if one core is touching the first of those 64 bytes and another thread is touching the last, from the hardware’s perspective there’s still sharing happening. “False sharing.” Thus, the Counter fix is using padding around the double values to try to space them out more so as to minimize the sharing that limits scalability.”

👏 File that under something I understand but would never have programmed.

“In the two benchmarks, we can see that the number of instructions executed is almost the same between when false sharing occurred (Index == 1) and didn’t (Index == 31), but the number of cache misses is more than three times larger in the false sharing case, and reasonably well correlated with the time increase. When one core performs a write, that invalidates the corresponding cache line in the other core’s cache, such that the other core then needs to reload the cache line, resulting in cache misses.”

Conclusion

“There are multiple forms of performance improvements covered throughout the post. Some of the improvements you get completely for free just by upgrading the runtime; the implementations in the runtime are better, and so when you run on them, your code just gets better, too. Some of the improvements you get completely for free by upgrading the runtime and recompiling; the C# compiler itself generates better code, often taking advantage of newer surface area exposed in the runtime. And other improvements are new features that, in addition to the runtime and compiler utilizing, you can utilize directly and make your code even faster. Educating about those capabilities and why and where you’d want to utilize them is important to me. But beyond the new features, the techniques employed in making all of the rest of the optimizations throughout the runtime are often more broadly applicable. By learning how these optimizations are applied in the runtime, you can extrapolate and apply similar techniques to your own code, making it that much faster.”

And that is much appreciated, Stephen. Having seen the available tools, I feel much better equipped to not only write but be able to advise on writing performant code.

The article Enumerated Science by Remy Porter (Daily WTF) describes a train wreck of a code example. It suitably illustrates why we really have to question whether scientists/juniors/etc. should really be writing code with so little training. If they wrote text this poorly, they’d be laughed out of their profession. Somehow, it’s perfectly fine to write code like this.

index = 0
for index, fname in enumerate(img_list):
    data = np.load(img_list[index])
    img = data[0][:,:]
    img_title 'img'+str(index).zfill(4)+'.jpg'
    cv2. imwrite(img_title, img)
    index = index + 1

The article points out all of the mistakes but I’ll summarize them here.

• Why does the code ignore the iteration item declared in fname? Instead, the code re-indexes into the array being iterated with img_list[index]. Like, why bro? You already had it! You know what img_list[index] is? It’s fname, bro.
• Why does the code bother calculating a complicated new filename that has nothing to do with the original filename? Why is the filename called img_title? It’s not a title; it’s a filename.
• Why does the code increment the index? It has no effect, does it? Or is it possible that this algorithm skips every other item? Honestly, why does Python even allow modification of the iterator variables? They should be const/immutable exactly so you can avoid doing something distracting like this.

This is a nearly 50-minute video about certain optimizations that used to be useful but which, in modern browsers, often get in the way of heuristic optimizations that browsers apply automatically.

How browsers REALLY load Web pages by We Love Speed / Robin Marx (YouTube)

“Preload should be applied with surgical precision”

• Specific edge cases (you really know what you’re doing)
• If the resource isn’t in the HTML

  • Fonts
  • Dynamic LCP images
  • JS imports

Basically, he said if you’re using preload, you’re almost certainly doing it wrong. For example, you can use fetchpriority=high on an img instead, and get the same performance benefit in the current crop of browsers.

These kinds of optimizations aren’t for most web sites. Most web sites have much larger performance problems than can be addressed with fetchpriority and preload optimizations. The one place where it’s still useful is for preloading fonts, which is a good idea for everyone.

While those two settings affect how the browser loads resources during the initial load o a page, setting loading=lazy on a resource takes it out of the initial load, so it puts it into a different part of the page-rendering (it’s loaded on demand, only when needed, e.g., when you scroll down to it).

The article The Best Programmers I Know by Matthias Endler seems almost too good to be true. NGL I feel seen. I have cited heavily from it, highlighting the parts I find especially interesting. At the end are a few pallid notes from me, but the meat of this article is the quote.

• Read the Reference
• Know Your Tools Really Well
• Read The Error Message

• Break Down Problems

  If you work as a professional developer, that is the bulk of the work you get paid to do: breaking down problems. If you do it right, it will feel like cheating: you just solve simple problems until you’re done.
   

• Don’t Be Afraid To Get Your Hands Dirty

  […] read a lot of code and they are not afraid to touch it. They never say “that’s not for me” or “I can’t help you here.” Instead, they just start and learn. Code is just code. They can just pick up any skill that is required with time and effort. Before you know it, they become the go-to person in the team for whatever they touched. Mostly because they were the only ones who were not afraid to touch it in the first place.
   

• Always Help Others

  Great engineers are in high demand and are always busy, but they always try to help. That’s because they are naturally curious and their supportive mind is what made them great engineers in the first place. It’s a sheer joy to have them on your team, because they are problem solvers.
   

• Write

  Most awesome engineers are well-spoken and happy to share knowledge.

  The best have some outlet for their thoughts: blogs, talks, open source, or a combination of those.

  I think there is a strong correlation between writing skills and programming. All the best engineers I know have good command over at least one human language – often more. Mastering the way you write is mastering the way you think and vice versa. A person’s writing style says so much about the way they think. If it’s confusing and lacks structure, their coding style will be too. If it’s concise, educational, well-structured, and witty at times, their code will be too.

  Excellent programmers find joy in playing with words.
   

• Never Stop Learning

  If there is a new tool they haven’t tried or a language they like, they will learn it. This way, they always stay on top of things […] the best engineers don’t follow trends, but they will always carefully evaluate the benefits of new technology. If they dismiss it, they can tell you exactly why, when the technology would be a good choice, and what the alternatives are.
   

• Have Patience

• Never Blame the Computer

  No matter how erratic or mischievous the behavior of a computer seems, there is always a logical explanation: you just haven’t found it yet!

  The best keep digging until they find the reason. They might not find the reason immediately, they might never find it, but they never blame external circumstances.

  With this attitude, they are able to make incredible progress and learn things that others fail to. When you mistake bugs for incomprehensible magic, magic is what it will always be.
   

• Don’t Be Afraid to Say “I Don’t Know”

  The best candidates said “Huh, I don’t know, but that’s an interesting question! If I had to guess, I would say…” and then they would proceed to deduce the answer. That’s a sign that you have the potential to be a great engineer.
   

• Keep It Simple

  Clever engineers write clever code. Exceptional engineers write simple code.

  That’s because most of the time, simple is enough. And simple is more maintainable than complex.

I disagree with the “Don’t Guess” one, in that I think “guessing”—forming a hypothesis—is the crux of scientific investigation. I think what the author probably meant was to “don’t leave a guess unproven.”

An addendum to the “magic” one above is that you should also know when to cut bait, i.e., when it’s not worth anyone’s time to find out what the real reason was. This can happen in one-off scripts, or in tight-deadline situations. Sometimes, you have to back-burner an investigation and either never bring it back to the front burner or learn the lesson at a later time.

But, yes, every problem solved is a bit of experience. It’s all worth it. A couple of decades of doing that and you might really have something.

This is a wide-ranging, occasionally delightfully foul-mouthed, and brutally honest interview with PM Maddy Mondaquila of Microsoft. Kudos to them for letting their best people do these kinds of informative and insightful interviews.

MAUI Lead Leaves to Work on .NET Aspire (and interview with Maddy Mondaquila) by Nick Chapsas (YouTube)

At 45:57,

“[…] yesterday Dave [Fowler] and I were fighting about if the Visual Studio .gitignore is getting dumber and he was like, ‘who cares about that? Why would anyone care about that?’ And I was, like, it’s 400 lines, dude. Like, we’re ignoring things from […] code-coverage tools that were deprecated five years ago. And then, finally, I start sending him screenshots, and he’s, like, wait, why is that in there? Why is that in there?”

At 49:00,

I absolutely loved the shout-out to “A Year without Santa Claus”. The plot summary was both accurate and possibly better than the actual movie (except for the musical number, which is worth the price of admission). I’m going to remember that Heat Miser vs. Cold Miser analogy. [1] Working in a company where most people didn’t grow up in the U.S. will make it an uphill battle to use it effectively, but I will not be discouraged.

At 51:49,

“Maddy MondaquilaThe more I’ve gotten to understand what customers are doing and talk to people and seeing the convoluted things that people do to develop an app, the more … I think I probably say once a week. I don’t know how anyone ships software. I don’t know how any of this stuff runs. This is all crazy to me, because everything is duct-taped together. Like, it is terrifying and you onboard someone and it takes like two weeks to get them to be able to run the app on their device.

“Like, what are we doing? What are we doing as a society? This is embarrassing.

“We should be able to do more than this and so that’s the thing about Aspire that excites me. We’re not trying to blackbox anything, right? We’re not trying to say, ‘oh, you use this and then your vendor-locked into this thing.’ It’s very much, like, we’re just trying to help you get off the ground and then you can grow out of it.

“I had done a lot with App Center […] and my fundamental issue with it […] was that you couldn’t grow up into a big-girl Azure service, is what I used to say. Like, once you hit the limits of apps, you had to start over and I was like, with Firebase or something, everything’s actually just gcp and when you’re ready you go into a big-girl [service], Google’s like, you’re ready, you move on.

“And so, Aspire was built with that in mind. Like, if you use Aspire for orchestration and then you use the client Integrations to do your databases, then at some point you’re, like, you know what? I actually don’t like the way that they’re setting this up. I’m going to do it my own way. You don’t rip anything out. You just keep going.

“And so that was like a really really big sell for me early. And then deployment was a whole other world that I did not understand and the more I’ve looked at it, I don’t, … again, I don’t know how anyone gets anything done. DevOps is insane. […] trying to bring that theory of, like, grow-out-able-ness instead of just replacing into deployment has been a very, very fun challenge to try and like tease apart.”

At 1:11:26,

“[…] we were talking to the Dutch police force and they are a completely polyglot shop so they have people running every language and there was one Java guy that came and and he was, like, so, like .NET’s, like, open-source and stuff now? And I was, like, yeah. And he was, like, but, like, really, like, it doesn’t have any ecosystem around it? And I was like what? YES and, like. there are real, like, expert, smart developers out there who just have no idea.”

This nearly deliberate ignorance about other programming languages, about tooling, about technique—it’s pervasive. There are people who care, and really want to find better combinations of tools and techniques to do their jobs better, to do what they love better. But there are just as many who just can’t even begin to imagine that there are other languages out there, that there are newer versions of the language they use available, with features that would actually be useful to them. These features are provably useful. They make your code more resilient, readable, and maintainable. They do not care. They don’t even know that they don’t care. They stopped learning a long time ago. Their curiosity is stunted.

It’s a pleasure watching people like Maddy and Nick discussing something that they’re passionate about. I’m passionate about that thing too, but it’s mostly because I understand that there is a good way of doing something—writing tests with MSTest and their bog-standard assertion library and no test-case-generation infrastructure—and a better way of doing something—writing tests with NUnit and their elegant assertion library, excellent error messages, and myriad ways of producing test cases. On the other hand, I keep a watchful eye on .NET’s new testing infrastructure that allows you to pre-compile unit tests as executables. This works for NUnit, too, so it’s a win all around.

That’s just one example but it sets the tone. People can’t explain why they don’t think they need ReSharper. They might not! But they have no idea why. They can’t explain why they use VSC instead of WebStorm. They have no idea that the latter actually supports a useful multi-file renaming refactoring whereas VSC still struggles to do a useful rename within a single file.

Everyone should be appalled and bitterly disappointed but, instead, they don’t even notice. They have no idea what they’re missing. So they don’t miss it. No wonder LLMs could catch up so quickly.

The article A 10x Faster TypeScript by Anders Hejlsberg (Microsoft) includes the following text, as well as a link to the video below,

“[…] we’ve begun work on a native port of the TypeScript compiler and tools. The native implementation will drastically improve editor startup, reduce most build times by 10x, and substantially reduce memory usage. By porting the current code-base, we expect to be able to preview a native implementation of tsc capable of command-line type-checking by mid-2025, with a feature-complete solution for project builds and a language service by the end of the year.”

This port is going to be available very, very soon, probably within the next three months. That’s quite ambitious.

A 10x Faster TypeScript by Anders Hejlsberg (YouTube)

The discussion C# vs. Go Concurrency Model (Reddit) led me to A 10x Faster TypeScript (HackerNews), which included a reference to Why Go? #411 (GitHub), which explains why Go was chosen,

“[…] the most important aspect is that we need to keep the new codebase as compatible as possible, both in terms of semantics and in terms of code structure. We expect to maintain both codebases for quite some time going forward. Languages that allow for a structurally similar codebase offer a significant boon for anyone making code changes because we can easily port changes between the two codebases. In contrast, languages that require fundamental rethinking of memory management, mutation, data structuring, polymorphism, laziness, etc., might be a better fit for a ground-up rewrite, but we’re undertaking this more as a port that maintains the existing behavior and critical optimizations we’ve built into the language. Idiomatic Go strongly resembles the existing coding patterns of the TypeScript codebase, which makes this porting effort much more tractable.”

The following image, included by a commentator, demonstrates quite nicely how idiomatically similar Go and TypeScript can be.

Go vs. Typescript code

If you read the rest of the justification, the similarities extend to the guts of the respective runtimes and their approach to memory-management and concurrency, but the visual illustration makes it much clearer that this is a port and not a rewrite.

A C# version—with its slightly different concurrency model and also a focus on byte-code rather than native code—would have involved much more change than this.

A version in Rust would have the focus on native-code generation but would have been a complete rewrite, as a lot of the concurrency and data-sharing possible in JavaScript would have to be explicitly allowed or worked around, something that you can’t always (or completely) hide with helper functions. The additional guarantees required in Rust to ensure safety would have to appear explicitly. Sure, you’d have the safety then, but it’s important to remember that, when you’re doing a migration, you should make sure you focus on one migration at a time.

Going from TypeScript to Go will improve some type-safety (though probably not even much) and massively improve speed with a native target. If you want the additional safety of Rust, then you’d do a separate migration step from Go to Rust.

There is another, longer interview video here:

TypeScript is being ported to Go | interview with Anders Hejlsberg by Michigan TypeScript (YouTube)

The interviewer is the guy who just published TypeScript types can run DOOM.
 

How to Add files to a Large Repository? (Reddit)

Git has opt-in support for handling large files.

• Use the –depth option to control how much history to clone (good for pipelines, where you’re usually only interested in the tip, so depth 1)
• Whereas depth controls how much you clone (size of the .git folder), sparse-checkout controls the size of your working tree.
• git logoUse LFS (Large File Storage) to store files. This will not remove large files from existing commits. This feature is seamless to enable and well-supported throughout the ecosystem.
• Once you’ve set up LFS for future commits, you can consider removing large files from already-existing commits using something like BFG and then re-adding them with LFS.

Recently, I saw that the following error had been fixed in a code review.

Classic primitive obsession error

The error shown above is an example of a design smell called Primitive Obsession. This is where code is “obsessed” with primitives, in that it uses a much “wider” type than is actually acceptable.

Whereas C++ has a typedef, TypeScript and Delphi Pascal have a type, C# has … nothing simple. The linked article describes a hand-coded version for making “narrower” types (e.g., MeanLength or ShortFiber). Our go-to generated-source guru Andrew Lock describes a solution that uses the StronglyTypedId package, but also links to a series from 2020 by Thomas Levesque that uses records.

It looks like you can use something like public record MeanLength(int Value); to succinctly define a narrower type. While it’s nice that C# autogenerates all the necessary machinery (equals, hashCode, etc.) for the record, it’s also unfortunate that it’s necessary, as we’re usually just trying to disambiguate two ints without further validation or restriction.

Also, I’m not recommending you leverage the type system to avoid primitive confusion in every code base! I’m just noting that the error that arose is so common that it not only has a name, but that there are well-defined solutions for avoiding that class of problems using the type system.

Before you start messing with these types in a language like C#—where, as noted, you don’t have a simple type-disambiguation mechanism—you need to have everyone in the team on board. Many programmers will consider them to be too heavy-handed (they suspect it affects performance somehow, and aren’t willing to trade any potential and unproven performance drawback anywhere for increased type-safety). Those are usually the same programmers who write code with a ton of primitive obsession and zero automated tests, so take their critique for what it’s worth.

The article Can You Get Better Doing a Bad Job? by Jim Neilsen cites Woody Harrelson as saying, “I think when you do your job badly you never really get better at your craft.”

Of course that’s true on the surface: If you manage to avoid learning anything else, then you will only ever get better at doing a bad job. The author expands on this point as follows,

“Experience is a hard teacher. Perhaps, from a technical standpoint, my skillset didn’t get any better. But from an experiential standpoint, my judgement got better. I learned to avoid (or try to re-structure) work that’s being carried out in a way that doesn’t align with its own purpose and essence.”

I agree with the highlighted bit especially. Any experience can be “good” because there is always room for seeing how you can make something useful in the midst of madness, how you can extract enjoyment out of even a poorly managed project. You can hone your programming skills; you can hone your diplomatic skills; you can learn how to turn it around. Maybe. At the worst, you learn how not to turn it around.

Dumpster fireBut then he writes that he “learned to avoid (or try to re-structure)” work that he doesn’t like. This is fine, but I would flip those two, to read “I learned to try to restructure (or, in the worst case, avoid),” in order to emphasize that someone should be trying to impart order to chaos—it might as well be you, if you’re so damned smart.

If everyone were avoiding bad projects, then where would good projects come from? Does everyone think that they’re so precious that good projects have to be prepared for them before they’ll even consider participating?

James MickensI recently stumbled upon some Essays from the funniest man in Microsoft Research by Raymond (Old New Thing). He is such a funny writer that this article, against convention and like the one before it (Brilliant articles by the funniest guy at Microsoft), will consist mostly of citations rather than an even mix of citations and paraphrasing that I naturally consider to be much more lucid and pithy. I quote at length to do the material justice, for documentation and to ensure that you all download the PDFs to see if there is more where that came from (there is). All emphases have been added.

He has since moved on to teach at Harvard University and is publishing things like The Wisdom of James Mickens there. That one’s about,

“Sometimes, when a professor goes on sabbatical, that professor will create a great book or sculpture, or commence upon a tour of the world to propagate some transcendental vision about the stars or aesthetics or our relationship to the divine. I myself spent my sabbatical writing a heavy metal album called “Ten Times Your Master: A King in Every Corner.” The ostensible conceit behind the album is that I, James Mickens, have gone missing, and the heavy metal community has formed a five-person tribute band called Ten Times Your Master to cover my songs. In reality, I, James Mickens, have not gone missing at all, and in fact am at the height of my powers, having created five distinct musical aliases to record a tribute album to myself, James Mickens.”

Which, you have to admit, is one of the most uniquely funny and interesting things that you’ve read in a while.

The notes below stem from a draft I wrote in 2014 and have only now published 11 years later. I have not altered them in any way because, well, I haven’t gotten any smarter, so who am I to correct my past self?

To Wash It All Away by James Mickens (Microsoft Research) (PDF) discusses the delusions of web programming:

“A modern Web page is a catastrophe. It’s like a scene from one of those apocalyptic medieval paintings that depicts what would happen if Galactus arrived: people are tumbling into fiery crevasses and lament- ing various lamentable things and hanging from playground equipment that would not pass OSHA safety checks. This kind of stuff is exactly what you’ll see if you look at the HTML, CSS, and JavaScript in a modern Web page. Of course, no human can truly “look” at this content, because a Web page is now like V’Ger from the first “Star Trek” movie, a piece of technology that we once understood but can no longer fathom, a thrashing leviathan of code and markup written by people so untrust- worthy that they’re not even third parties, they’re fifth parties who weren’t even INVITED to the party

“In a rational universe, a single uncaught excep- tion would terminate a program, and if a program continued to execute after throwing such an exception, we would know that Ragnarok is here and Odin is not happy. In the browser world, ignoring uncaught exceptions is called “Wednesday, and all days not called ‘Wednesday.’” The JavaScript event loop is quite impervious to conventional notions of software reliabil- ity, so if an event handler throws an exception, the event loop will literally pretend like nothing happened and keep running. This ludicrous momentum continues even if, in the case of the seventh error, the Web page tries to call init() on an object that has no init() method. You should feel uncomfortable that a Web page can disagree with itself about the existence of initialization routines, but the page is still allowed to do things with things. Such a dramatic mismatch of expectations would be unacceptable in any other context.

“it would definitely be horrible if your browser’s scripting lan- guage combined the prototype-based inheritance of Self, a quasi-functional aspect borrowed from LISP, a structured syntax adapted from C, and an aggressively asynchronous I/O model that requires elaborate callback chains that span multiple generations of hard-working Americans. OH NO I’VE JUST DESCRIBED JAVASCRIPT.

“JavaScript is dynamically typed, and its aggressive type co- ercion rules were apparently designed by Monty Python. For example, 12 == “12” because the string is coerced into a num- ber. This is a bit silly, but it kind of makes sense. Now consider the fact that null == undefined. That is completely janky; a reference that points to null is not undefined—IT IS DEFINED AS POINTING TO THE NULL VALUE. And now that you’re warmed up, look at this: “\r\n\t” == false. Here’s why: the browser detects that the two operands have different types, so it converts false to 0 and retries the comparison. The operands still have different types (string and number), so the browser coerces “\r\n\t” into the number 0, because somehow, a non-zero number of characters is equal to 0. Voila—0 equals
0! AWESOME.

“ I obviously get what I deserve if my JavaScript library redefines native prototypes in a way that breaks my own code. However, a single frame in a Web page contains multiple JavaScript libraries from multiple origins, so who knows what kinds of horrendous prototype manipulations those heathen libraries did before my library even got to run. This is just one of the reasons why the phrase “JavaScript secu- rity” causes Bibles to burst into flames.

“Some JavaScript libraries intention- ally begin with an initial semicolon, to ensure that if the library is appended to another one (e.g., to save HTTP roundtrips during download), the JavaScript parser will not try to merge the last statement of the first library and the first statement of the second library into some kind of semicolon-riven statement party. Such an initial semicolon is called a “defensive semico- lon.” That is the saddest programming concept that I’ve ever heard, and I am fluent in C++.

“I could go on and on about the reasons why JavaScript is a cancer upon the world. I know that there are people who like JavaScript, and I hope that these people find the mental health services that they so desperately need. I don’t know all of the answers in life, but I do know all of the things which aren’t
the answers, and JavaScript falls into the same category as Scientology, homeopathic medicine, and making dogs wear tiny sweaters due to a misplaced belief that this is what dogs would do if they had access to looms and opposable thumbs.

“The first log entry says that the browser executed a downloaded file as JavaScript, even though the MIME type of the file was text/html. Here’s a life tip: when you’re confused about what something is, DON’T EXECUTE IT TO DISCOVER MORE CLUES. This is like observing that your next-door neighbor is a creepy, bedraggled man with weird eyes, and then you start falling asleep on his doorstep using a chloroform rag as a pillow, just to make sure that he’s not going to tie you to a radiator and force you to paint tiny figurines. Here’s how your life story ends: YOU ARE A PAINTER OF TINY FIGURINES.”

This World of Ours by James Mickens (Microsoft Research) (PDF) discusses the delusions of passwords, cryptography and security research:

“Basically, you’re either dealing with Mossad or not-Mossad. If your adversary is not-Mossad, then you’ll probably be fine if you pick a good pass- word and don’t respond to emails from ChEaPestPAiNPi11s@ virus-basket.biz.ru. If your adversary is the Mossad, YOU’RE GONNA DIE AND THERE’S NOTHING THAT YOU CAN DO ABOUT IT. The Mossad is not intimidated by the fact that you employ https://. If the Mossad wants your data, they’re going to use a drone to replace your cellphone with a piece of uranium that’s shaped like a cellphone, and when you die of tumors filled with tumors, they’re going to hold a press conference and say “It wasn’t us” as they wear t-shirts that say “IT WAS DEFI- NITELY US,” and then they’re going to buy all of your stuff at your estate sale so that they can directly look at the photos of your vacation instead of reading your insipid emails about them. In summary, https:// and two dollars will get you a bus ticket to nowhere. Also, SANTA CLAUS ISN’T REAL. When it rains, it pours.”

“Even worse than the PGP acolytes are the folks who claim that we can use online social networks to bootstrap a key infra- structure. Sadly, the people in an online social network are the same confused, ill-equipped blunderhats who inhabit the phys- ical world. Thus, social network people are the same people who install desktop search toolbars, and who try to click on the monkey to win an iPad, and who are willing to at least enter- tain the notion that buying a fortune-telling app for any more money than “no money” is a good idea. These are not the best people in the history of people, yet somehow, I am supposed to stitch these clowns into a rich cryptographic tapestry that supports key revocation and verifiable audit trails. One time, I was on a plane, and a man asked me why his laptop wasn’t working, and I tried to hit the power button, and I noticed that the power button was sticky, and I said, hey, why is the power button sticky, and he said, oh, IT’S BECAUSE I SPILLED AN ENTIRE SODA ONTO IT BUT THAT’S NOT A PROBLEM RIGHT? I don’t think that this dude is ready to orchestrate cryptographic operations on 2048-bit integers.”

“I realize that, in an ideal world, I would recycle my trash, and contribute 10% of my income to charity, and willingly accept the cognitive overhead of finegrained security labels. However, pragmatists understand that I will spend the bulk of my disposable income on comic books, and instead of recycling, I will throw all of my trash into New Jersey.”

“Similarly, we know that IFC research should not focus on what would happen if I somehow used seventeen types of labels to describe three types of variables. Instead, IFC research should focus on what will happen when I definitely give all my variables The God Label so that my program compiles and I can return to my loved ones.”

This is the enduring problem of security. This is what always happens when the people charged with providing security aren’t well-versed enough to handle the complexity of the chore. Our job as framework developers is to make a system that is as simple as possible, but no simpler. Sometimes, initial efforts miss the mark. Sometimes, there is no way to feasibly bridge the gap between what users’ capabilities and the complexity of the tasks that they have to solve.

“The worst part about growing up is that the world becomes more constrained. As a child, it seems completely reasonable to build a spaceship out of bed sheets, firecrackers, and lawn fur- niture; as you get older, you realize that the S.S. Improbable will not take you to space, but instead a lonely killing field of fire, Child Protective Services, and awkward local news interviews, not necessarily in that order, but with everything showing up eventually.”

“Security research is the continual process of discovering that your spaceship is a deathtrap. However, as John F. Kennedy once said, “SCREW IT WE’RE GOING TO THE MOON.” I cannot live my life in fear because someone named PhreakusMaximus at DefConHat 2014 showed that you can induce peanut allergies at a distance using an SMS message
and a lock of your victim’s hair. If that’s how it is, I accept it and move on. Thinking about security is like thinking about where to ride your motorcycle: the safe places are no fun, and the fun places are not safe.”

A sample from Tenure announcement: April 2019 by James Mickens (Harvard School of Engineering & Applied Sciences):

James Mickens’s Harvard tenure announcement is unorthodox. He’s basically trash-talking to his enemies, who can no longer touch him. Some samples:

“My seventh-favorite enemy is obviously Alan Fontaine of Iowa State University. I know that you’re reading this, Alan, because you keep inviting me to NSF panels even though I mailed you a glitter bomb that was shaped like me mailing you a glitter bomb. Your theories on Muppet physiology are childish and naïve, and I viciously refute them in my upcoming article “Parasitic Infections of Muppet Gastrointestinal Hand Holes.“ I wish you the best when you lose your endowed professorship and are forced to teach at a lower-tier institution that can’t even afford real Muppets and has to use oven mitts with faces drawn on them.”

“On that page, I used my grandfather’s charcoal pencil to draw a picture of myself dunking a basketball over your confused, athletically-incompetent body;”

I recently wrote Real quick on MVVM and now I see that a good colleague and friend has written his own MVVM understandings by Austin Jones (Austin's Journey for Meaning). His piece got me thinking again about how the concept is a good start but isn’t really sufficient.

Justifying the view model

Somewhere near the beginning, he writes,

“The View Model’s function is separate from the Model. Abstraction requires discipline to not let two pieces of code that do the same thing become the same thing, purely out of convenience. Things that operate together should be functionally coupled, not just that same code.”

While I deeply appreciate the sentiment, I think that (A) most people are going to be unconvinced that they need additional complexity for such a vague goal, and (B) there are more concrete reasons to keep them separate. In Real quick on MVVM, I posited a simple example, repeated below.

record Person(
  string FirstName,
  string LastName,
  Company Company,
  DateTime BirthDate);

The view model might want to expose:

int Age => DateTime.Now.Year − _model.BirthDate.Year;

string FullName => $"{_model.FirstName} {_model.LastName}";

Company Company { get; }

IReadOnlyList<Company> AvailableCompanies { get; }

The AvailableCompanies is for the drop-down menu.

The data in the model is a different shape than that required by the view. This happens quite quickly and quite often. Anyone who tries to “cheat” by using a type as both a model and view model will quickly be writing spaghetti code.

It is the view-model’s job to marshal data to and from its own shape to that of the model. It is decidedly not the model’s job to do that, because it exposes data, while one or more views might display it in different ways. Perhaps another view is showing the birthdate directly, in which case that view model would simply pass the value through unmodified.

Thinking through an example

“Most logic seems to fall into the View Model as your business logic rules are often mirrored by presentation rules. E.g. a button has to be disabled if the user hasn’t met some requirement.”

It may seem too picky but I would instead use the verb reflect instead of mirror, to say that the view model exposes properties that reflect the state in the model. Just off the top of my head, I can imagine that each component of the architecture has unique duties, as illustrated in the example below,

• A model contains several properties that must adhere to certain rules in order to be saved.
• A validation service determines whether those rules have been satisfied, returning a list of zero or more validation results.
• A view model exposes the most recent list of validations as a property, as well as a property called readyToSubmit. The view model triggers the service to calculate a new list of validations when the view notifies it that a relevant change has been made.
• A view binds the validations as it sees fit—either attaching properties directly to the controls that will display their values and allow users to manipulate them, as well as exposing the list of validations to the user in some way—as well as binding the Enabled property of the submission button to the readyToSubmit property.

This is just a simple example but we can see that the model is just a data container. In classic OO, the service would have been part of those objects. However, it’s far more flexible to keep the model as a set of “dumb” DTOs and to keep the logic in the service. This makes it much easier to replace the validation logic in specific cases, without touching the data layer, which doesn’t need to change.

The view model does the work of managing calls to the validation service as well as retaining the results as long as the view needs them. The view model doesn’t know anything about buttons. It doesn’t need to know that they can be enabled or disabled. That’s the view’s job, which deals with the actual representations presented to the user.

This makes the view model, in turn, flexible enough to be used with alternate representations. For example, we can imagine a view or view model that simply auto-saves when readyToSubmit is true, so it would have been a shame to have named that property saveButtonEnabled because it would have been an awkward fit for the hypothetical second view.

Benefits

As you can well imagine, it’s incredibly easy to test systems built in this way, as you can very easily construct the data/model that you want and test something like the validation service. You can also very easily build on top of that to verify that the view model updates and notifies as expected. You can even bind to its properties to verify that a potential view would have received the expected notifications.

The view doesn’t have more logic in it than binding. Views are more finicky to test—although it’s not impossible or even especially difficult with practice, its it’s also not usually necessary. For most problems that crop up, your tests eliminate the possibility that the bug is anywhere other than the view, so you quickly find where the incorrect binding. Obviously, if errors like this are chronic—or if you have very complex views—then you’ll want to test the view with end-to-end tests. Just remember that testing the view usually requires the most effort, results in the slowest tests, and provides the least benefit, so you should really be doing those last, if at all.

Conclusion

My colleague’s example focuses more on how the service layer pertains to persistence, for loading and storing models. I wanted to provide an example that doesn’t have anything to do with persistence but shows that there is non-persistence logic that obviously—at least in hindsight—doesn’t belong anywhere but in the service layer.

I’ve been working with this type of abstraction since at least 2002, when I started working on the Atlas framework at Opus Software AG, which was written in Delphi Pascal. We didn’t call it MVVM but we had a very clear separation between the object model, the view model, and renderers.

The relatively short post My Washing Machine Refreshed My Thinking on Software Effort Estimation by Chris Horsley (Cosive) is kind of interesting, in that it’s a cautionary tale about being overconfident about your estimates. As the title suggests, his was a real-world task where he’d assumed that a tenth iteration would go just as smoothly. He draws some good conclusions but for what I think might be the wrong reasons.

“[…] while 90% of the project will be the same, there’s going to be one critical difference between the last 5 projects and this project that seemed trivial at the time of estimation but will throw off our whole schedule. It could be one or all of:”

1. Our well-used task-running framework we were going to use for a relatively small part of the system is totally unmaintained now and we’d have to fork it to make it fit for purpose again.
2. Our entire development tooling ecosystem was obsoleted 18 months after the last time we did this, so we’re going to be learning the sharp edges of a whole new toolchain from scratch.
3. We find that our OS version has moved on and no longer supports key requirements for our existing dependencies, requiring rethinking or developing from scratch.
4. We need our infrastructure stack to use one component we’ve never used before and it doesn’t work anything like we expected.

The things that he lists as stuff that you generally don’t think about are exactly the kinds of things that you should keep in mind when doing an estimate, though! Like, if you’ve been doing estimates and any of these things are a surprise to you, then you’ve just been LARPing at estimates.

Even when you copy/paste an existing solution to a similar problem, you have to consider the context in which the original was developed and the degree to which that context might be different this time around. It’s not easy but it’s your job to be aware of limitations and concessions at all times. Always consider that the passage of time is part of the context.

It too so long to set up his washing machine because he’s a rank amateur at doing that, despite having done it so many times. He got lucky the first nine times because literally nothing that could go wrong went wrong. He thought that his experience promoted him to a senior-level position but he was still a junior-level monteur. On his tenth time, everything went wrong and he was totally blindsided by it—but only because he’d learned nothing about the system he was working on.

He didn’t learn, for example, what his requirements or environmental expectations were nor that he should quickly check to verify that they were satisfied before he started.

It’s as if he’d gone downstairs to check his car’s oil but hadn’t brought his house keys with him because the door to the garage had always been propped before. When he had to go back upstairs to get his house keys, that was considered a blindsiding showstopper that you couldn’t have accounted for.

Even after the first setback, he didn’t sit back to take stock of the situation and plan his next few steps rather than just his next step. That’s the classic misinterpretation of agile: it doesn’t mean you should turn your brain off in planning. It doesn’t mean you shouldn’t do any long-term planning. It means that you should always be prepared to change your plan. It means that you should take a little time to plan for the immediately foreseeable work. That doesn’t mean that you’re suddenly doing waterfall! It means you’re filling your backlog.

The tweet ”Programming” by Andrej Karpathy (Twitter) is what some people are calling the future of programming—with the loudest claiming that it’s already here and that you’re all missing the boat if you’re not programming like this.

“There’s a new kind of coding I call “vibe coding”, where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. It’s possible because the LLMs (e.g. Cursor Composer w Sonnet) are getting too good. Also I just talk to Composer with SuperWhisper so I barely even touch the keyboard. I ask for the dumbest things like “decrease the padding on the sidebar by half” because I’m too lazy to find it. I “Accept All” always, I don’t read the diffs anymore. When I get error messages I just copy paste them in with no comment, usually that fixes it. The code grows beyond my usual comprehension, I’d have to really read through it for a while. Sometimes the LLMs can’t fix a bug so I just work around it or ask for random changes until it goes away. It’s not too bad for throwaway weekend projects, but still quite amusing. I’m building a project or webapp, but it’s not really coding − I just see stuff, say stuff, run stuff, and copy paste stuff, and it mostly works.”

The people who claim that this represents the future of all human engineering effort are mostly fools and mountebanks, who are either not software developers or engineers, or who claim that they are, but never really understood what we do.

Karpathy is, at least, very explicit that he’s doing this for “throwaway” projects. He is building, at best, prototypes and, at worst, inscrutable toys that don’t do what he wants, and he’s not going to bother trying to fix them.

This style only works because Karpathy already kind of knows how to program and he’s smart. This style will not make someone magically able to produce anything other than a crude facsimile of what has already been produced before. They will likely get stuck in myriad cul-de-sacs where the LLM avows very confidently that something will work and it will not work—sometimes subtly, sometimes overtly.

The other day, I was confronted three times with outright inaccurate, or simply outdated or inapplicable suggestions from Copilot. I did not request these suggestions; they were introduced by colleagues during reviews and collaboration.

Clinging to the past

In one case, someone asked in a PR whether we really could eliminate the framework declaration in app.config files.

<configuration>
  <startup>
      <supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.8" />
   </startup>
</configuration>

The Microsoft documentation still confidently declares that you need to include this declaration but says nothing about why it’s not included in any of its starter templates.

Copilot therefore confidently writes that you absolutely need the declaration for reasons that might have been somewhat applicable in 2009, or maybe even 2012.

The app in question works with .NET Framework 4.8, a version that has been available for three years. There is no follow-up version in sight. A 4.x version of the framework has been included on all versions of Windows for years. The app doesn’t run with a lower version. It already declares this in the <TargetFramework> element of the project file.

We don’t need to artificially pin the version with an outdated mechanism. The version of the framework give in the project file is not only sufficient, but adding an outdated alternative that says something slightly different but will be ignored is distracting, to say the least. It’s a recipe for configuration cargo cults and future goose chases, which is why I threw it out. Copilot would chirpily encourage people to put it right back in, and to waste time fine-tuning it.

Simplistic and incorrect

In another case, a colleague was having trouble upgrading Entity Framework from using the System.Data.SqlClient to using the Microsoft.Data.SqlClient instead. There was an error where a registration wasn’t being honored and the app couldn’t find the new provider.

The Copilot solution was to confidently recommend adding a whole bunch of crap that wouldn’t solve the problem in any sane manner—and couldn’t. It just amounted to vague instructions to include the EF framework—which my colleague was obviously already doing.

The solution to the problem was going to have to be more subtle, but my colleague was getting lost in the weeds of trying to figure out the subtlety underlying what Copilot might possibly mean with its naive suggestion.

I am continually amazed by how much leeway people give these LLM tools, very optimistically assuming that a given tool “understood” the intent and is suggesting a brilliant solution that will reveal itself if only the developer can rise to the occasion.

When I was called in, I read the error message and it was pretty clear that something in the system was overriding the settings we wanted to use. It turned out that there was a reference to the old data provider in one of the core libraries that we’d upgraded.

Instead, we ended up focusing on the fact that the EF support took a default dependency on a relatively old 5.x component but the latest version was 6.x. The suspicion was that perhaps something would work better in 6.x. When my colleague added the newer version to all projects, things started working again, so apparently a bug had been fixed. But which bug?

While it was good to have a workaround, adding the reference everywhere was an awkward solution. So my colleague took a small timebox to investigate further

He eventually discovered—without Copilot—that there was a registration for the old data provider buried in our library code. Removing that single line of configuration in the base library solved the error. Including another registration shouldn’t cause the configuration to ignore subsequent registrations of other data providers, though. I think that the 5.x version wasn’t capable of properly managing multiple, registered data providers but that the 6.x version could. When the library was forced to use a newer version, everything worked again.

The maintainable solution that we ended up using was:

1. Remove the unwanted configuration entry in the base library.
2. Add a single reference to the newer version of the transient library to ensure that we were using the 6.x version—with the bug fix that we technically no longer needed—in all consumers of the library. This step wasn’t even strictly necessary but there’s no harm in it. We marked the package-inclusion with a reason and a link to the work item that was tracking the work and included a write-up of what had happened.

A solution that doesn’t exist

Finally, I had updated a build-pipeline template to accept a new parameter: OutputFolder, which was to default to a value based on the value of another parameter SolutionFolder. The following solution feels like a natural way of expressing this intent.

parameters:
    SolutionFolder: ''
    OutputFolder: '${{ parameters.SolutionFolder }}\Output'

This is what it looked like in the actual template:

Copilot's bold suggestion

However, if you try it, you will get an error message saying that “A template expression is not allowed in this context”:

Template Expression not allowed

According to the machines—the screenshot below is DuckDuckGo, which is based on ChatGPT by default—it will cheerily tell you that you can totally do exactly what you clearly can’t do.

AI thinks parameters can reference each other

This is the cloud theory link referenced in the screenshot. There’s nothing in there that has anything to do with the question or its answer. The post is about Azure pipelines but it’s a pretty banal article that explains the basics and it will utterly waste your time if you were looking for a definitive answer on how to reference parameters from one another, as the LLM claims is possible.

The best-case scenario is a mediocre solution that works.

The worst-case scenario is a productive-looking rabbit-hole that leads nowhere.

This is my worry about this whole LLM-based tool thing: it’s wasting people’s time and making them inefficient. The best-case scenario is if it manages to give them a mediocre solution that works. The worst-case scenario is that they get routed down a productive-looking rabbit-hole that leads nowhere but which they can then claim that had been productive because they’d been doing “work” and who can blame them if it was the LLM that started them on that path?

How do you fix this for real? You use the solution that I came up with after quickly learning from Allow parameter defaults to be calculated based on other parameters #2737 on GitHub that the feature I was looking for had never been implemented.

I used a variable and coalesce, as shown below.

variables:
    defaultOutputPath: '${{parameters.SolutionFolder}}/Output'
    templateOutputFolder: ${{ coalesce(parameters.OutputFolder, defaultOutputPath) }}

Curious, though, I went to Copilot to find out what it would suggest (assuming that Copilot is a bit more advanced than the default ChatGPT used by DuckDuckGo). As usual, it returned a lot of boilerplate text that may or may not have actually corresponded to the code it provided. I didn’t bother reading most of it and just kept the code examples.

parameters:
  − name: repositoryName
    type: string
    default: ''

  − name: sonarQubeProjectKey
    type: string
    default: ''

jobs:
  − job: SetDefaults
    steps:
      − script: |
          echo "##vso[task.setvariable variable=sonarQubeProjectKey]com.company.${{ parameters.repositoryName }}"
        displayName: 'Set SonarQube Project Key'

This is an impenetrable and unmaintainable disaster that uses a script to write a variable to the job’s terminal, which also sets it in the pipeline environment as a side-effect. Why would you do it that way when you just need a local variable? I already know what the solution is, so I know that I can keep prompting instead of wasting my time trying out this version, which I think might work but which is not elegant, legible, or particularly maintainable..

That’s the advantage of being a senior developer: I will know a good answer when I see it, so the LLM can save me time as I pull the slot-machine arm until it comes up sevens.

In the brusque style I typically use with LLMs, I pressed on.

parameters:
  − name: repositoryName
    type: string
    default: 'default-repo'

  − name: sonarQubeProjectKey
    type: string
    default: ''

variables:
  − name: sonarQubeProjectKey
    value: ${{ parameters.sonarQubeProjectKey != '' && parameters.sonarQubeProjectKey || 'com.company.' + parameters.repositoryName }}

jobs:
  − job: Build
    steps:
      − script: |
          echo "SonarQube Project Key: $(sonarQubeProjectKey)"
        displayName: 'Display SonarQube Project Key'

That’s a bit better. Now there’s a variable declaration, even though it still uses the unfortunate old-style name/value syntax that is not super-intuitive. It also uses a bizarre bastardization of the || operator to coalesce to a default value if it’s not set. I’m not an expert on pipeline-expression syntax but the placement of the && looks pretty wonky.

As with the first answer, I didn’t even bother trying to see if that one-liner mess even compiles, to say nothing of getting me what I want. Even if it works, it’s an impenetrable and unmaintainable line that no-one will ever dare touch again.

Since I already knew the answer, I told it straight-up how I wanted it to write the expression.

parameters:
  − name: repositoryName
    type: string
    default: 'default-repo'

  − name: sonarQubeProjectKey
    type: string
    default: ''

variables:
  sonarQubeProjectKey: $[coalesce(parameters.sonarQubeProjectKey, 'com.company.' + parameters.repositoryName)]

jobs:
  − job: Build
    steps:
      − script: |
          echo "SonarQube Project Key: $(sonarQubeProjectKey)"
        displayName: 'Display SonarQube Project Key'

It’s unclear why it’s using Runtime expression syntax instead of the more-standard Macro syntax variables. It doesn’t matter, though because, according to the documentation (and as far I knew already), you have to use the Template expression syntax when referencing template parameters (which we most certainly are—it’s right there in the question).

Is it complicated that there’s a table as well as a section called What syntax should I use? that explains which form to use where and what to watch out for? Yes, it is. But I don’t make the rules. I just play by them.

The tool of the future that you’re a fool and knave for not using because junior devs super-powered by it will blow by your ancient ass if you don’t learn how to use it

Copilot, on the hand, is not constrained by petty rules, and would have cheerily helped a less-savvy developer piss away an entire afternoon if not a couple of days chasing phantom errors, trying desperately to get the code produced by the genius machine to work as advertised. The thing about pipeline scripts is…you have to execute them in the cloud. The turnaround time is murder.

Let’s take a quick peek at my version, the one I had before we starting playing around with the tool of the future that you’re a fool and knave for not using because junior devs super-powered by it will blow by your ancient ass if you don’t learn how to use it.

variables:
    defaultOutputPath: '${{parameters.SolutionFolder}}/Output'
    templateOutputFolder: ${{ coalesce(parameters.OutputFolder, defaultOutputPath) }}

Succinct, easy to read, with useful variable names. Easy, peasy, lemon-squeezy. Also, it works! That sets it apart from the Copilot suggestions above.

The only trouble I had in the PR was that my reviewer didn’t know what the coalesce function did…and the top German translation was for Verbinden, which means to link, tie, or join. That meant he thought it was a synonym for concatenate, which it most certainly is not. It was a good learning opportunity, where I reminded him of the coalesce function in SQL, as well as the null-coalescing operators—?? and ??=—in C#.

Look, the world is complicated, human languages and creations are messy, and a glorified Plinko machine like an LLM doesn’t seem quite yet cut out for helping us navigate it.

This is a 40-minute discussion about the combining the latest technologies, like scroll-snapping, scroll-driven animations, anchoring, etc. to produce responsive, progressive, animated, modern, and very fast sites without any JavaScript at all.

Pure CSS Scroll Spy Table of contents − No JavaScript Required! by Kevin Powell & Adam Argyle (YouTube)

Carousels

Adam uses it all to build carousels, which is fine for demos and proving the power of the technologies, but … I’m not a fan. While our two hosts mention that Netflix comprises only carousels, they don’t really discuss that Netflix is also a deeply unsatisfying experience for finding content.

Slow adoption rate

They finish up with an interesting discussion of how quickly changes are introduced and the absolutely legitimate reasons why adoption of some features is so slow. It’s often difficult for developers to be aware that a feature exists and also to be aware that it would be a solution for the problem that they’re having. There’s also the fact that most developers and product owners will allow the scope of the familiar to limit their vision of what is possible.

You really need people who stay on top of these things and can say that yes, it is possible to animate this now, or it is possible to eliminate a ton of cruft here, and also to be aware of whether that feature is available on all target platforms, or whether it can be made optional with progressive enhancement, or … it’s a very complicated, complex thing to handle. It takes years before a feature is just known and accepted. Often, it takes a new generation of programmers who’ve grown up with that feature to know how to use it.

Forget about yesterday’s cruft!

Just think: today, you can build responsive, progressive, fast, pretty, and accessible web sites with no layout hacks and no JavaScript. Everything just works. But you haven’t always been able to do that, so there is a large percentage of the web-developer community that is not aware that this is the case because they stopped paying attention a while ago and are stuck on the feature set that they know.

At best, they’re aware that a feature exists but wasn’t ready for primetime when they last checked, even though they’ve not checked in a while. Even if they’re aware of it, they might not have the time or budget to use it in existing projects, where everything has already been tested. Who’s going to risk ripping out a ton of custom code to replace it with two lines of CSS, when you have to test everything all over again?

In November of 2024, Microsoft released a lot of videos about .NET to accompany the release of .NET 9. I watched/listened to a lot of these, with varying levels of attention paid. When something caught my attention, I took notes. These videos are roughly in the other than I watched them, although I reserved the right to shuffle them about a bit to improve grouping.

AI

1. AI Building Blocks − A new, unified AI layer by Steve Sanderson
2. Discover the Latest GitHub Copilot Features for .NET Developers in Visual Studio
3. Building and scaling cloud-native, intelligent applications on Azure and .NET

Aspire

1. What’s new in .NET Aspire

ASP.NET

1. Bulletproof ASP.NET Core APIs: The OWASP API Security Top Ten
2. What’s New for ASP.NET Core & Blazor in .NET 9
3. Exploring the New Fluent UI Blazor Library: Next-Gen Web Components and Architectural Innovations
4. I Confronted Microsoft About Blazor’s Future
5. Introducing HybridCache in ASP.NET Core
6. Easily Improve Web Application Performance using .NET 9 Caching and Redis
7. Clean Architecture with ASP.NET Core 9
8. New tools in Visual Studio for Web API developers

Desktop

1. Modern WinForms Development with .NET 9
2. MVVM Building Blocks for WinUI and WPF Development
3. How Fidelity uses .NET MAUI for Cross-platform desktop
4. Cross-Platform Magic: Transforming WPF Apps with Avalonia XPF

General

1. Advanced Pattern Matching in C#
2. Testing.Platform, the new way to run .NET tests
3. Modernising Legacy .NET Codebases with NDepend
4. F# − Nullable Reference Types
5. Embedding Python libraries in .NET services and applications

AI

AI Building Blocks − A new, unified AI layer by Steve Sanderson (YouTube)

The API looks very approachable and straightforward to use. I wish that they would stop papering over the inaccurate responses, though.

At about 17;45, he writes that the additional pair of socks has been added to the cart and that it’s “gone up in the way that it should.” Except that’s not what the response showed. The response showed the total number of pairs of socks in the cart, yes, but it showed the price only for the additional pair of socks that was added in the last step. It noted that this was the case but it was quite confusing to show the total number of items in the cart and then write “the total price for that pair is”, which would confuse a reader into thinking that it was the total for the cart, unless they read carefully. Using language like “total” for a single item is confusing, if not misleading.

This is the kind of stuff that would ordinarily be cleaned up in a final product but it’s doubtful that this kind of stuff will be cleaned up in AI-generated prototypes that are scaled up to production, especially since people that will know how to fix it might become rarer. It will be deemed “good enough” and the inexorable erosion of software quality will continue.

Also, Sanderson had GitHub Copilot running during the entire demo and he pretty much completely ignored all of its suggestions, choosing instead to copy/paste pre-written snippets. This is fine, of course! It’s just that, … why didn’t he just turn off the annoying prompts that constantly plastered completely irrelevant information into his video? It was quite distracting.

Discover the Latest GitHub Copilot Features for .NET Developers in Visual Studio (YouTube)

I suppose this is future of programming? In this video, you can watch a young person ask an AI to add usings because you have no idea what they are and don’t know that the IDE could just add them for you automatically. She described everything as “awesome” and thought that the interaction loop was super-intuitive and easy to use, as she typed out natural-language command after natural-language command to try to get the machine to do what a programmer could have done in seconds with analysis-based tools.

I suppose if you want to program without knowing anything about the technologies, then this is probably going to get you a little bit further. Maybe. It was pretty painful to watch, though, like someone claiming that they were building a house by throwing wood at bags of cement and hoping for the best.

Building and scaling cloud-native, intelligent applications on Azure and .NET by dotnet | Scott Hunter & Paul Yuknewicz (YouTube)

This is a longer presentation—almost an hour—that goes in-depth on converting an ASP.Net Web application first to an Azure Function, and then adding a .NET Aspire Host project to not only coordinate communication between the front-end client and the Function project, but also to facilitate deployment directly to a Cloud Container.

Hunter explains the different between WithReference(), which indicates that a service depends on another service being started and WaitFor(), which extends the reference to indicate that a service should wait until a health-check of its dependency indicates success before declaring itself available. If you think about it, almost all references are important in his way, but .NET Aspire still makes the distinction to give your app flexibility in starting up. If you have two service that depend on each other, they can’t each wait for each other, or you might be a deadlock (unless the health-check can return success before the service itself is ready).

On top of that, there is a method called WithExternalHttpEndpoints(), which they describe as “doing the right thing” and setting up a virtual private network in the cloud container so that only the web client has access to the Azure Function endpoint. “It’s network-isolated by default, which is one of the features of container apps, which is the default way of publishing in an Aspire application.” This is very cool and seems a lot easier than writing a bunch of custom Bicep code.

The web client can now access the Azure Function at the alias that it assigned in the .NET Aspire host project’s configuration. That configuration is all written in C#, with a nice fluent API. I’m a little disappointed that they don’t use a shared C# constant to reference the unique strings, but nobody in any of the .NET Aspire demos seems to do that, preferring to ride the ragged edge of disaster with copy/pasted identifiers.

Getting back to the demo: He shows how it’s published to Azure, using Managed Identity. The .NET Aspire dashboard shows the remote resources with full logging available, also available and published to Azure. It’s the same dashboard as you would use locally.

Everything they demonstrated is available in several solutions, all listed in Samples for Building and Deploying Cloud Native Intelligent Apps by Paul Yuknewicz (GitHub Gist).

They say, “All you do is clone the repo, and then you do azd init and azd up and they’re super-easy to get in the cloud and try yourself.” It will deploy the resources into your subscription and region of choice. There are detailed instructions for each example, e.g., azure-functions-openai-aisearch-dotnet (RAG example).

Paul goes into this example in more detail, examining the Bicep scripts (which I’m not sure whether they’re hand-written or generated by .NET Aspire). They cover how to build an embedding for RAG and how much support there is in .NET now for making this kind of thing easy. They use a standard HTTP body as input but discuss many other potential input streams (queues, etc.), all of which are just as easily supported by default.

I learned that “agenting” is just the cool new term for allowing an LLM to use “tools”. It’s also called “skilling” (adding “skills”). The most approachable epithet is the least-cool-sounding one: “function-calling”, which at least explains what it does. So “agenting” is empowering an LLM to execute tools in order to enhance results.

Like many of the other videos, they use the .http Files feature to store recipes of calls to make against an HTTP server.

When their demo doesn’t initially work, you see how Microsoft also has a locked-down network that they have to work around. They very smoothly transitioned to Scott discussing security initiatives that Microsoft has taken and is taking to lock things down by default for new applications, including those generated by customers. Secure by default. They then smoothly transition back to the demo, for which Paul has reset his network to a working state. Nice job.

The funny thing about some of these demos, though, is that they used an LLM to find out what time it is in New York City…and everyone held their breath to see if it would get it right. It took about 10 seconds to figure it out. That’s laughable on its face, but the point is that it’s now quite easy to set up a powerful tool to round-trip to an LLM running securely in Azure, built with access to custom functions (agenting) and custom data (RAG). Subsequent demos are more impressive.

At 40:00, Paul addresses why it’s interesting to solve problems in this way: you can scale up quickly to much higher data volumes without changing the architecture or implementation at all. This is great but you want to remember the aphorisms, “you’re not Google” and “you don’t have big data.” In this case, though, an argument can be made that these technologies are the right way to build it when its small and when its large. The support and abstractions are good enough that you don’t have to choose a a non-scalable solution early in the process to save money.

On the subject of ignoring warnings in code, though, Paul had a typo in the word “cacluated” that Visual Studio showed him and he still hadn’t bothered to quick-fix it, even though the entire world would be seeing it.

The final example is showing how to use a “blog trigger”, which is a function that reacts when new data is added to an Azure blob container. When you drop a PDF into it, it uses standard, available recognition tools to analyze the document and then funnel that content to the LLM (which is not great at “cracking” PDFs provided as context on its own). These dependent tasks are captured as “activities”, which are composed as part of a “durable function”, which is essentially a high-level abstraction on top of potentially distributed calls.

This is the part that Paul called “orchestration” at the beginning of this section. It’s not orchestration like Kubernetes (although possibly related, way down at the low level), it’s orchestration of high-level activities and representing them as a single function call that takes an indeterminate amount of time. Paul demonstrates how much tooling and web-based observability there is available for debugging and monitoring solutions.

“Again, this is Functions, so you can do this at scale. You can send million of documents.”

It just costs money, but you’re not otherwise limited by the architecture if you build it with these concepts, this architecture, and these building blocks.

Aspire

What's new in .NET Aspire by Damian Edwards and Maddy Montaquila (YouTube)

This addition to the .NET ecosystem continues to impress me. While this video demonstrates only .NET Aspire, the previous video (Building and scaling cloud-native, intelligent applications on Azure and .NET) also made heavy use of Aspire. Many of the videos would include Aspire, actually.

ASP.NET

Bulletproof ASP.NET Core APIs: The OWASP API Security Top Ten by dotnet / Christian Wenz (YouTube)

This is a very useful introduction to common security issues and how to address them. He talks about how to program by default so that the issues never come up.

At around 19:00, he even discusses how to build a threat model. He kind of backs into describing it by talking about the types of risks for which you might need processual mitigations. That is, the threat model talks about something like “the system allows a single user to book multiple seats for themselves on a plane” and then talks about (A) whether you even want to mitigate this and (B) which kinds of mitigations would work against it.

What's New for ASP.NET Core & Blazor in .NET 9 by dotnet / Daniel Roth (YouTube)

This was pretty informative, overall. I wish he’d spent a bit more time on HybridCache, which seems like a big win.

Oh, hey, look at that: there’s a video for it, called Introducing HybridCache in ASP.NET Core.

Exploring the New Fluent UI Blazor Library: Next-Gen Web Components and Architectural Innovations by dotnet / Vincent Baaij & Denis Voituron (YouTube)

This library is quite nice and seems to offer a good basis on which to start projects. It’s interesting, though, how people just say things that they’ve heard. Denis said at one point that Playwright will be integrated soon in order to improve “code quality”, which is absolutely not what tests do. Tests do the second thing he mentioned: “avoid regression.” If you have tests, your code can be any old quality. The tests don’t care, as long as they’re green.

I Confronted Microsoft About Blazor's Future by Nick Chapsas & Daniel Roth (YouTube)

I mean, honestly, can you just write “interviewed” instead of “confronted”? Do we really have to write everything as if it were a title in a Fleet Street broadsheet? It’s a friendly interview.

• They discussed the difference between .NET LTS and non-LTS versions. There is no difference in quality.
• Why wasn’t there so much Blazor news this year? The .NET 8 release was so huge that they spent a lot of time in the first half of the year after the release simply stabilizing that release, and then focused on quality improvements in .NET 9. There is a big feature that consolidates the different ways of using Blazor into a more uniform concept.
• Who’s using Blazor? How big is it? Year-over-year growth is high but the overall usage numbers are still kind of low, relative to other frameworks, like the by-now ancient WebForms, ASP.Net MVC, ASP.Net Core, etc.
• Is Blazor going to go the way of Silverlight? No, it’s the recommended way to develop web sites on .NET. Nick gave a good intro here, talking about how good WebForms actually was—taking aside ViewState—but that frameworks like Next.JS are still re-inventing what WebForms had already offered and pawning it off as a revelation. It’s kind of how most of the server-side frameworks now just look like PHP.
• .NET Aspire has a super-short support cycle; it goes obsolete with the first point release. You have to upgrade it rather aggressively to stay in-support. It’s not part of the .NET release. It’s out-of-band. It’s super-useful but it’s a bunch of tools and wiring without much of its own API. It makes sense to keep support cycles short because, while it’s been released for others because it’s so damned useful, it’s also acknowledged that the surface will potentially change quite a bit as more and more real-world use cases appear.

  Why do we even need Blazor? Microsoft isn’t using it anywhere, is it? The problem that Blazor solves is trying to build a web site with a team that doesn’t know any of the languages, tooling, or paradigms of front-end development. For the vast majority of web sites, you really don’t need full-fledged React or Angular or Svelte. While there are developers who can legitimately live in both worlds, Blazor is for those for whom a good web site is good enough.

  You can make anything in Blazor, of course, but it really helps you get to a standard, good-enough view (especially with Blazor Fluent UI) that covers so much of the software being built. Roth describes how the client-side world is such a different beast and that spinning developers up to be productive and happy in that world takes a lot of time, money, and resources.

  It’s a fair point. Many people just can’t wrap their heads around that style of development. It’s too alien to them. I would venture to say that most web developers aren’t very good at software-development, don’t really understand the environment or their tools, and are just cargo-culting their way to freedom and happiness.

  It’s why we’ve had so many RAD dev environments, it’s why we have so many frameworks. Even after a good 20 years of development churn, things are still churning. There is almost no consensus on how to address the plethora of non-functional requirements in clients: accessibility, compatibility, graceful degradation, progressive enhancement, etc. The best philosophy seems to be PWA and probably an MPA not an SPA for apps, which is not most web sites. Most web sites are mostly static. So the common web frameworks are ill-suited to those kinds of pages. Etc. Etc. “Blazor lets me get more with less. I can’t afford to hire a full-time front-end developer.”

• Why is .NET trying to do everything? Isn’t that a recipe for being mediocre in most things? “I think it’s fair to say that there are parts of the .NET stack that have … more strengths than other parts.” I think that there’s a real need at Microsoft for doing a lot of things in all areas. MS needs to develop web sites, needs to develop mobile apps, needs to develop cloud-based apps, etc. They’re going to develop at this scale anyway. We can be happy that this workman’s version of these tools are available, and cross-platform. Apple makes some amazing technology that only works on their hardware and on their systems and they don’t have a cloud. .NET runs on AWS, GCP, Azure, etc. .NET runs on ARM, x86, Linux, Windows, MacOS, etc. The base library is incredibly well-designed.
• How big is the Blazor team? There are six full-time devs. It sits atop other parts, like SignalR, which has two full-time devs. The actual framework is six engineers. He mentions that the community does some heavy-lifting here as well.
• Who’s using Blazor? Roth mentions some customers, and then talks about how it’s used in a lot of places internally at Microsoft. They don’t use Blazor for Teams, Office365, large customer-facing products. Those use React. A lot of that is historical, because Blazor has only been around for 5 years. They used to use Script#, which was a transpiler for C# to JavaScript, but then they moved to React. A ton of the recent, internal LOB-style products use Blazor. That’s what it’s for. Smaller teams use Blazor and there are thousands of devs who use Blazor at Microsoft.

• Is there anything public-facing that uses Blazor? Very little. The Aspire Dashboard is one of the only things. Part of that reason is “technology fit”. Blazor is very good for internal LOB that runs on a known set of devices and capabilities. Think Office, though: they need to be able to run on anything. That’s a completely different proposition because it constrains you more. You need more control of the stack. A high-level solution like Blazor doesn’t save you time there; it costs you time. For products that need to be optimized in terms of download-size and speed, etc., then you probably should use JavaScript directly.

  They recommend Blazor if it fits your scenarios. Otherwise, use ASP.Net Core with a JS front-end. He made the comparison to using Node on the server. It’s not the optimal thing for performance, but it might match your team best. But you can scale the server-side with money. You can’t scale the client-side.

  Roth agrees: if you have millisecond-initial-download constraints, then Blazor isn’t for you. He does say that you’d be surprised how many apps aren’t like that.

  I’m not surprised. He says that even the heavyweight Blazor server model, which is basically PHP-style, if we’re honest, then you can support dozens of thousands of concurrent users on a single, modestly sized VM. Most apps have expectations of hundreds of concurrent users, at most. Being server-based will restrict your interactivity if you rely on it too much with server-based stuff. For forms and LOB, though? It’s fine. You’re not Facebook. Relax.

• How many people use the web-assembly stuff instead? Quite a few, actually. The server-side Blazor is slightly more popular, though. They’re both growing at about the same speed. .NET 10 plans to invest more in the server-side version. They need to solve some problems about server-side state: hydration strategies for longer-lived processes and workflows.
• What if you had to make an app for millions of users? If it’s B2B or LOB, then consider Blazor. Start with Server-side, then move to interactive server-rendering on the client, move to WASM-based to push individual islands to the client where necessary. If it’s customer-facing, then it’s going to be .NET on the backend and a JavaScript front-end (he doesn’t say “React”, notably). I think MS engineers are also seeing the value in writing to the web platform, using JavaScript. He doesn’t even say TypeScript, because they’re so close these days. It would be amazing if browsers allowed the syntax directly so we didn’t have to transpile anymore. There’s a proposal for this.
• Yeah, but which platform would you use if you couldn’t use Blazor? TypeScript because static typing is awesome. He mentioned that the Angular 18 release was “pretty compelling”. I personally looked into it based on this and couldn’t figure out what he was talking about; it looked like an incremental upgrade to me.

• Daniel asked what about server-side rendering? Would you look at Next.JS? SSR? Server-side components? Nick says Next.JS but I don’t agree. I think their solutions, just like Remix, about which I’ve also read quite a bit. Their solutions get … complicated at scale, with their attempts to paper over the difference between client and server parts without being forced to know where anything is running tending to be quite leaky abstractions. I’ve read quite a bit about Remix and Next.JS and, in both of those, I’ve seen where cracks show that people deeply familiar with the technology think “aren’t so bad” but that’s only because they know that it used to be so much worse.

  Daniel says “Look around at what is going to be around for a while. Everybody has to plan for the longevity of their career.” This is so sad, though. We don’t get anything great from people “planning for the longevity of their career.” We get cool things from people who just can’t help but try to make something better, to make something cool, to make something to help themselves, that interests them.

  Dude, React is a hype. Most people are using it poorly. You get to leaky abstractions in the first two days of teaching, where you have to tell people how to avoid horrible performance with useMemo() and useCallback(). They’re working on a compiled version of React, which is just where Svelte has always been.

• What’s next for Blazor? Roth talks about SSR a lot. Interaction between SSR and client-side, etc. Performance and caching. I think Blazor will be a better, more well-thought-out and much less ad-hoc approach to SSR than Next.JS and Remix have gotten.

  Why? Because the people that Microsoft has and the culture that they have tend to produce really good APIs. That’s just a fact. Multi-threading feature is on the radar again. They’re going to try again, but it’s not committed yet. Security is the #1 push right now, though. So if Blazor has work to do there, then that takes priority. This will result in a more secure stack for users of Blazor as well.

• If you had to work with a different back-end language, what would you use? Daniel responded “Python”. Nick said “Kotlin”, which he says is how JetBrains fixed Java by making it C#. I would take another look at Swift, which I haven’t used for anything real since version 5. Or maybe finally do something in Rust, just to see where the tooling is at.

Introducing HybridCache in ASP.NET Core by dotnet / Marc Gravell (YouTube)

This video covers Redis, HybridCache, and stampede-protection.

Easily Improve Web Application Performance using .NET 9 Caching and Redis by dotnet / Catherine Wang & Matt Soucoup (YouTube)

Moar caching. Pretty good.

Clean Architecture with ASP.NET Core 9 by dotnet / Steve Smith (ardalis) (YouTube)

I follow this guy’s blog and find many of his articles to be a bit basic. It feels like he’s writing just enough articles to keep his Microsoft MVP badge. In this video, though, he is en fuego, absolutely ripping through a whirlwind introduction to clean architecture, with some demos and some code. You can find the sample projects in NimblePros / eShopOnWeb (GitHub) and ardalis / CleanArchitecture (GitHub).

He also very quickly demonstrates how to use the new API window to submit requests to the running server; this replaces external tools like Postman, keeping you within Visual Studio (or Rider, which has supported this for even longer).

In his full-tilt presentation, he also mentions using Papercut-SMTP, which is “a 2-in-1 quick email viewer AND built-in SMTP server (designed to receive messages only).” This is ideal for local devs to test emailing code and can be easily integrated and started with .NET Aspire.

After having generated a solution using his clean-architecture template, he says that “this is the slowest part … opening the new solution in Visual Studio,” but this is really unfair because the solution is restoring and doing everything in the background, while he’s clicking around in the solution explorer. Previous versions of Visual Studio would never have allowed this. He even launches the product and is browsing around in the Aspire Dashboard and the web server’s OpenAPI front-end within seconds. It’s an impressive demo.

New tools in Visual Studio for Web API developers by dotnet / Sayed Ibrahim Hashimi (YouTube)

Sayed starts off with a good overview of the basics of .http files, which showed up with .NET 8 and allow you to keep sets of API calls for testing, much in the way that people have been using Postman or Insomnia. The new feature is that .http files can now store values in variables for transferring results from one call to others. He uses Visual Studio but, as I’ve noted elsewhere, Rider has supported them for even longer than Visual Studio.

He also shows a not-quite-ready-for-primetime-but-coming-soon feature of Visual Studio called the Endpoints Explorer, which is a sort of Swagger/OpenAPI browser available as a Visual Studio panel.

Desktop

Modern WinForms Development with .NET 9 by dotnet / Merrie McGaw & Klaus Loffelmann (YouTube)

This was quite an interesting video, in that it really drives home that WinForms is here to stay. The community pushed hard to make a lot of the code base expose and use nullability. Microsoft has also improved performance in System.Drawing and replaced all interop with code generated by CSWin32. There is also improved support on base UI objects for asynchronous calls like Form.ShowAsync and so on.

I like that his demo to show text in a color-mode-aware manner failed because he was creating the brush with the right color but he wasn’t assigning it anywhere. How do I know he wasn’t using it anywhere? Because Visual Studio had grayed out the instance variable to which he had initialized his brush. He’d assigned the brush but hadn’t actually assigned it to be used by any control. This is why you configure and then pay attention to the warnings and suggestions in your IDE, folks. It really does help you solve otherwise pretty hairy problems. In this case, I was able to diagnose his problem just from a brief flash of less than a second of him scrolling through his file.

It hurts me so much to watch people click toolbar buttons to comment/uncomment code. Seriously, you’re fired.

So, cool video but this guy really shows how outdated some WinForms developers are. 😹

MVVM Building Blocks for WinUI and WPF Development by dotnet / Sergio Pedri & XAML Llama (YouTube)

I learned that the latest preview version of the MVVM Community toolkit is already using partial properties—which are new to C# 12 in .NET 9—to help you write even less code for your view models. Also, you can use x:Bind instead of x:Binding to have a compile-time, reflection-free binding, which has much better support for code-completion, inspections, etc.

How Fidelity uses .NET MAUI for Cross-platform desktop by dotnet / David Ortinau, Matthew Faust, & Kevin Bieri (YouTube)

This is a pretty impressive demo of what is now possible with Maui. They’re using Telerik controls. The neat thing about this Maui app is that it runs on MacOS as well. They talk about WebViews a lot—and painting to the canvas—so it also runs in a browser. He does discuss how using SkiaSharp is a valuable place to seek performance but that you are almost certainly going to make usability or accessibility concessions. Use source generators, not reflection. Interestingly, they mention how you can speed things up with Windows Defender by signing assemblies with “HSM methodology”, which is something to look into, I think.

I just can’t help but think that it isn’t any faster or better-looking than the trading app that I was lucky enough to be able to build for Peak6 back in 2010–2013. That one was a multi-threaded Winforms app that connected to a data hose that shoveled tons of data toward the app—dozens to hundreds of thousands of refreshed data rows per second—in dozens of open windows and portals. Each of the grids showed data in grouped, tiered, aggregated, and heat-mapped views. When it really got going, you had 40+ open windows on eight screens, all updating in real-time. Close the app and re-open it and it came right back where it was. it was very, very colorful.

Cross-Platform Magic: Transforming WPF Apps with Avalonia XPF by dotnet / Mike James (YouTube)

This is a great look at Avalonia, which is a very interesting migration path from the Windows-only WPF to a multi-platform approach that doesn’t involve rewriting everything in Maui or Blazor.

With Avalonia, you can either port from WPF, primarily changing the styles to use the more CSS-style styling of Avalonia. Because of this feature, though, an Avalonia app can relatively easily be deployed to a web application. He discusses a community project Avalonia Visual Basic 6 by BAndysc (GitHub), which you can browse in a demo.

It runs in a browser, it runs on all desktop platforms—including Linux, which Maui doesn’t support.

On top of that, it also supports something called Avalonia XPS, which is a complete replacement for the WPF rendering engine, so you can “port” an app just by changing the SDK uses in the project files. That’s it. He demonstrates it live and it works extremely quickly and seamlessly. Of course, if you have P/Invokes or a lot of custom rendering—or external components that aren’t compatible—then you’ll have to do more work. But it’s a huge step forward to getting WPF apps running on other platforms.

For charts, he mentions that SciChart is the “best” charting library for WPF and that it is compatible with Avalonia. They use the XPS layer to provide support for Linux platforms.

His final demo shows a WPF app (the calculator) running with XPS but targeting a web browser. This is wild. I had completely underestimated Avalonia.

General

Advanced Pattern Matching in C# by dotnet / Oli Sturm (YouTube)

At about 8:30, he shows a nice usage of switch expressions with range expressions to make a recursive summing function.

static int Sum(Span<int> l) => l switch {
    [] => 0,
    [var x, .. var xs] => × + Sum(xs)
};

Nice!

There were a lot of interesting examples in this video. The final one for refactoring a red-black tree was really cool. It’s funny how bad these people are at demos, though. He showed us that he had 513 tests running and passing in 0.5s. Then he says that the passing tests is the only thing that’s important. Um, no, it’s also important that rebalancing is done in a reasonable amount of time, so we should also keep an eye on the time the tests take with any refactored implementation.

To demonstrate that the tests actually test the code he’s going to refactor, he wiped out the enter implementation and re-ran the tests. But they didn’t run because he was no longer returning a value from his method, so it didn’t even compile. He blew right by that and said “see, the tests don’t run.” Um, no, the program no longer compiles and you haven’t proven anything about the connection between the implementation you’re going to refactor and the tests.

All that aside, though, it’s quite an elegant solution that looks just like the original Haskell code. It’s not legible at a glance but is a very succinct representation that uses the standard style for these kinds of things.

Red-black tree in C#

Testing.Platform, the new way to run .NET tests by dotnet / Jakub Jares (YouTube)

The Testing.Platform is very nice. It promotes test suites to first-class citizens, built as executable files without any reflection-based assembly-scanning at runtime. Instead, the source generator scans and generates code for running the tests. It also supports a “watch” mode (called “hot reload”, of course), which lets you keep the tests running as a separate app. It’s much faster and more reliable, it’s AOT-friendly, etc. etc.

When it was initially introduced in January of 2024, the only drawback was that you could only use it with MSUnit. That’s changed! At 23:00, he shows how to enable and run NUnit-based and XUnit-based tests with Testing.Platform. I really, really like how MS-based projects like this embrace open and non-Microsoft standards: the testing platform is to replace VSTest, which only ran on Windows. Testing.Platform is platform-agnostic and testing-framework-agnostic, bringing its Reflection-free, AOT-capable, runtime-stable approach for everyone.

He demonstrates running a solution with NUnit, XUnit, and MSUnit tests running as a standalone, collectively with dotnet run, and in Visual Studio. Rider / VSTest writes “JetBrains Rider can run tests from any custom test framework that uses VSTest or Microsoft.Testing.Platform.” ReSharper / VSTest writes “ReSharper can run tests from any custom test framework that uses VSTest or Microsoft.Testing.Platform.”

Good news all around. This thing is ready to be used!

Modernising Legacy .NET Codebases with NDepend by dotnet / Goh Chun Lin (YouTube)

I really like the tool NDepend and I was quite interested to see how Lin used it to upgrade a codebase from .NET Standard 2.1 to target .NET 8, and then .NET 9. He’s from Singapore so he was not easy for me to understand—but he definitely knows what he’s doing.

This video reminded me of a time about 9-10 years ago when I was really heavily using the tool to modernize the Quino code-base, which had grown quite organically and was proving difficult to use for only web servers, especially those running on Linux. We made our own journey from .NET Framework 4.7.2 to .NET Standard 1.0 (didn’t work at all), then to .NET Standard 2.0 (success!). I continued to use the tool for the next five years.

Here is a list of related articles, which I argue go into more depth on how to use NDepend than the video does. NDepend hasn’t changed a significant amount in 10 years—it was an incredibly powerful tool then, and it still is. I haven’t used it much at my new job at Uster but time will tell.

• 2014: The Road to Quino 2.0: Maintaining architecture with NDepend (part I)
• 2014: The Road to Quino 2.0: Maintaining architecture with NDepend (part II)
• 2015: Splitting up assemblies in Quino using NDepend (Part I)
• 2015: Iterating with NDepend to remove cyclic dependencies (Part II)
• 2018: The Road to Quino 2.0: Maintaining architecture with NDepend (part I)
• 2019: Finding deep assembly dependencies

F# − Nullable Reference Types by dotnet | Petr & Tomáš Grošup (YouTube)

This is by far the nerdiest video in the entire series. It’s chock-full of interesting information about F#, with a focus on the new support for null, which it has in order to better interoperate with .NET modules built in other languages that do support null. F# is a functional language and uses options. However, Tomáš demonstrates how this is not sufficient when working with data coming in from outside of the F# system. The feature piggybacks on the |-operator to allow | null in any type definition.

Most of the rest of the presentation shows how the new feature integrates with options, pattern-matching, generic types, etc. There are analyzers in the compiler that help your code shed “nullness” as soon as possible, leaving most of your F# code without nulls, as God intended.

Tomáš calls “shadowing” a feature, which is being a bit generous. It’s a nice trick to declare a “new” version of the incoming argument that has a type narrowed by a null-check function call. Languages like TypeScript and C# actually have a “feature” in which the type is narrowed without variable-shadowing. It amounts to the same thing, though. I suppose F#’s version is less gimmicky and implicit, but shadowing is frowned upon in so many other places, because it’s super-confusing when done inadvertently. Using shadowing to narrow a type is a clear use case but it will also prevent analysis from being able to preclude accidental shadowing.

This video is mostly just an F# tutorial, though. At the end, a guy named Kinfey pops in to tell Tomáš that he better wrap things up. It’s quite unprofessional but also quite funny.

Embedding Python libraries in .NET services and applications by dotnet | Anthony Shaw & Aaron Powell (YouTube)

This isn’t the most organized demo, and they don’t really show how to set things up but it does show how integration of Python into .NET is much simplified by source generators that generate bindings for marshaling data to and from Python.

Not only that but .NET Aspire is indispensable for configuring a system like this, not only for tying together the moving parts—PostgreSql, Python API, web front-end—but also for monitoring not only the startup but also API executions, which you can track in a nice process-graph for each request (shown below).

.NET Aspire for CSnakes / Python

It even shows how the chart is rendered in Python, returned as bytes to C# and then rendered into the body of the response directly (basically sending back an image rendered in Python without conversion).

Unlike previous attempts like IronPython, this approach uses .NET Aspire to simplify integration of Python projects and code without changing it. It just integrates it, like taking care of setting up the PostgreSql database and then passing the connection string to the Python code.

Anthony discusses at the end how the common data types used in Python ML processing (tensors, etc.) are all supported in an efficient manner, allowing you to pass buffers back and forth from Python to .NET and offloading code like web servers and GUIs to .NET development while benefitting from existing Python libraries.

I imagine that this is just an interim solution until the .NET ecosystem catches up more with Python—which it is definitely going to do, if it hasn’t largely done so already.

This is a nearly 100-minute-long interview and discussion about programming-language design and evolution. It gets deep into the weeds on very specific and relatively advanced language features. While a feature may eventually feel quite simple to use, the considerations about how to design it and how to fit it into the landscape of the rest of the language can be very, very complex. There are a lot of moving parts to consider in a language, runtime, community, and ecosystem as established as .NET and C#.

The Exciting Future of C# with Mads Torgersen by Nick Chapsas (YouTube)

Existential Types

At about 45:00, Mads describes a higher-level typing system that allows for “hiding” some generic-type parameters.

“There are various features that are related but not quite the same. There are ‘associated types’ in some languages and there’s what one scholar calls ‘abstract types’ [I think he meant ‘existential types’], which might be my favorite version of the feature, which are kind of an alternative to generics or a kind of generics.

“The really short version is they help you not have so damn many type arguments all the time, yeah? Okay; essentially think of it as a class. Instead of a class having a type parameter saying, I’m an animal with a type argument saying which kind of food it eats…that means every time you talk about animals, you have to pass type arguments around. That’s really annoying because what kind of food it eats is inherent to it. It shouldn’t be like something on the outside; it should be a member, saying my food type is [whatever] …

“…and if you do that, then you can kind of tamp down on a lot of the…sometimes you just end up in generics overload, or passing the same stuff around. And every one of these related types has a type argument for which particular implementation of the other related interfaces it is using and they all carry the same five type arguments around all the time.

“So that would be a feature, if we can get it right, and if we can work it into the runtime, and it’s limited—and we are occasionally talking about it—that could be a really really beautiful and quite impactful addition to C#.”

It is described in detail in Proposal: Existential types for interfaces and abstract types #8711 (GitHub). While Mads’s description of it made my ears perk up, the proposal does a better job of explaining it.

An interface might look like this:

interface ICounter<protected T>
{
    T Start { get; }
    void Next(T current);
    bool Done { get; }
}

Describing the visibility of the generic parameter with the keyword protected is part of the proposal.

Since the parameter is protected, external users would use the interface without the generic parameter, like this:

void M(ICounter ic)
{
    var × = ic.Start;
    while (!ic.Done)
    {
        × = ic.Next(x);
    }
}

Only an implementation is required to provide a type parameter:

class Counter : ICounter<int>
{
     int Start => 0;
     int Next(int current) => current + 1;
     bool Done => current == 42;
}

Type unions

At 58:00, they discuss what something like “discriminated unions” or “tagged unions” would look like in a decidedly object-oriented language like C#.

“Mads: We arrived at this degree of clarity around what our options are. The type unions—use of the word ‘types’ there reflects that one conclusion that we reached is that, in C, unions should be ‘unions of types’. If you look at F# or other functional languages, discriminated unions are not unions of types. They’re unions of something with a name, a tag-discriminator, whatever you want to call it, the tagged unions that can then be deconstructed to give you values of one or more types. So the different options are like ‘named options’ but they’re not things in and of their own right. They’re just a means to get to what’s inside.

“In C#, one thing that we agree on is that that has to change. It has to be that the things are types. So you don’t have to pattern on a union and get an A or you get a dog and immediately you have to decompose it into how many legs it has and and what it eats. You know what its name is. You can carry it around as a dog. It makes sense in its own right and it can be its own object. So that’s essentially trying to take an object-oriented view on what discriminated unions would look like in C#.

“Nick: Fundamentally, discriminated unions are a very functional-programming-like concept and they don’t fit in a language that already has inheritance right? Because the idea is it’s their version of inheritance.

“Mads: Exactly.”

Design mistakes in C#

At 01:22:00, Nick asks what are Mads’s three least-favorite/most-hated features of C#, things that he wishes had never gone in or that, in a better world, he would remove.

1. Events, because they should never have been a language feature. They should have been a runtime/library feature, at most.
2. Delegates, because they were forced to be collection types because of events. Their execution dynamics are indeterminate (because of ordering) and they actually break covariance (kind of like arrays).
3. Void should have been a type (as it is in Typescript and Eiffel). Its being a language feature means that they bifurcate all other support, like requiring a distinction between Func and Action.
4. dynamic is a pretty great feature academically but the value of the feature ended up being much less than they’d hoped. “Performance-wise, it’s a disaster. There’s a whole bunch of infrastructure to maintain it. It doesn’t carry its own weight.”

Nullability

The final part of the discussion is about nullability-checking in the compiler, and its limitations and strengths. “The point is that it’s so much better than nothing.” The feature makes you think about what even should be allowed to be null.

My advice is to avoid returning or accepting null unless you absolutely can’t avoid it. Don’t ruin your API, of course, but be absolutely sure that null needs to be an option. It is much easier to write code to APIs that never return null. Consider using sentinel objects instead. The feature in C# has gotten a lot better and is very good now. init and required properties helped finalize the feature.

The article Do JavaScript frameworks still need portals? by Ollie Williams (Fully Stacked) explains a bit more about the relationship between dialog, popover, and anchor as well as how these elements have made “portal” support in frameworks obsolete. The following ~20-minute video shows how to use only CSS and HTML to position and display elements, regardless of their nesting in the document.

CSS Popover + Anchor Positioning is Magical by Kevin Powell (YouTube)

The main example uses CSS anchor positioning (MDN), position-try (MDN) and :popover-open (MDN). You can use position-try to direct the browser to adjust the appearance of the popover when there isn’t enough “space” on-screen at its default position. You can also animate everything.

Popover menu

.user-button {
  padding: 0;
  border-radius: 100vw;
  aspect-ratio: 1;
  anchor-name: --profile-button;
}

.profile-menu {
  position: absolute;
  position-anchor: --profile-button
  top: anchor (bottom) ;
  right: anchor(right);
  margin: 0;
  inset: auto;
  margin-block-start: 6px;
}

.profile-menu:popover-open {
  display: grid;
}

@position-try --bottom {
  inset: unset;
  top: anchor(bottom);
  right: anchor(right);
}

Some of the features described in the video below sound pretty interesting but they also sounds super-complicated, with deferred-loading, hydration, signals, and event/replay all combined with decisions about whether something loads on the client or the server or whether it’s initially built on the server but then enhanced on the client and then run independently from the server, …

I understand that a lot of this technology is for optimizing large web sites, but It’s honestly questionable who can actually take advantage of it. That’s always been the problem with trying to massively optimize your web applications as if you were Google. It’s nice that this stuff is available; it’s unfortunate that so much of it has to be opt-in because the cognitive load required to use it well is very, very high. It’s the same experience I’ve had in React. When teaching it, you get into the weeds on performance and useMemo() and useCallback and profiling very quickly, which makes it…not really a beginners library.

Take a look at the video, though. Maybe something piques your interest. Be warned, though, the presentation style is…interesting, to say the least. It’s got a “definitely not robots” vibe to it. More below the video.

Angular v19 Developer Event (YouTube)

Man, I thought some of the recent MS .NET videos seemed less-than-professional but those were downright charming compared to whatever this thing from Google is. All of the people featured in this video are so stiff, so obviously touched-up and coifed, that it’s throwing me off.

• The guys are hold their arms on-camera so that you can see that they go to the gym.
• Everyone does that super-annoying “I’ve taken an oration course and have learned to artificially move my hands to emphasize my points.” that people who are definitely not robots do.
• The one (only?) lady’s diction reveals to me that she definitely filmed her segment a dozen times.

The “jokes” fall so flat that I can’t even imagine which audience would think this is funny.This video feels like it belongs in a high school. Every line is scripted. There’s no spontaneity and it’s not good. I’m used to presenters who act much more naturally than this; even people like Kevin Powell and Nick Chapsas are still obviously performing, it doesn’t feel like a “don’t do drugs” PSA put on by parents.

The following video shows not only when and how to use calc-size(), it also mixes in advice on generating timing functions for animations, sprinkles CSS variables throughout, and even uses overflow: clip combined with an absolutely positioned element to reveal more content without disturbing the layout.

Animate and do math on things like height: auto with interpolate-size and calc-size() by Kevin Powell (YouTube)

The syntax for calc-size() is, as Kevin says, “weird”; you have to pass two parameters: the first is the name of the logical size you’d like to use, while the second parameter is a formula that uses the placeholder size, which accepts the value of the first parameter. In a sense you are passing the argument, along with a lambda that accepts that argument.

The following CSS sets the inline (horizontal in LTR and RTL) size of an element to be whatever the intrinsic size of the element would have been, given the size of its children, plus 3rem (where rem is the font size of the root element (MDN)).

inline-size: calc-size(max-content, size + 3rem);

This video accompanies Kevin’s article for the 12 days of code 2024, if you’d rather read about it, with more code examples.

The picture and title are, as usual, clickbait-y, because apparently people don’t click on videos that sound educational unless you promise them ground-breaking learnings. Still, I don’t hate the player; I hate the game. But it’s the world we have.

The video is quite informative and is 90% not the guy pictured. Instead, it’s another guy called Gui Ferrera, who is quite competent.

The Correct Way to Run Database Migrations in EF Core by Nick Chapsas / @gui.ferreira (YouTube)

He starts by explaining how to deploy migrations in production—you don’t just run them, as you would in development. Why not? Because, if you run the migrations as part of your application startup, then your application implicitly has permissions to modify the database schema—permissions that you are unlikely to revoke or downgrade.

As an alternative, you can use the dotnet-ef tool to generate an ef migrations bundle (he names the file efbundle), which is an executable that you can then just run, using the pipeline secret that has administrator access to the target database. This executable runs separately and is only in charge of migrating the database to a particular version. Your application will run and fail if the schema is not correct, which is the desired behavior. If it is correct, your application will run with a database user with much lower permissions—at the very least, it won’t be able to issue DDL commands.

The bundle option generates a binary; there is also a script option, which generates SQL. This is pretty neat and there’s even a flag called idempotent, which allows you to generate a script that will ensure that previous migrations have been applied before continuing with subsequent migrations.

The implementation isn’t as obviously straightforward as it advertises itself. There must be limitations for custom-migration behavior that uses program logic. I know, because Quino [1] had a very similar feature and, although we could generate SQL for some user customizations to the migration process, there was no way to support everything.

It’s nice to see how solid the EF migrations story has gotten, even though I think the design still suffers when switching branches. You need much more developer discipline to keep your local database usable and in-sync. Anecdotally, I hear that most developers just trash their local database all the time, and rely much more on seeding functions to restore the state. You tend to lose your local ad-hoc-created data and it takes a bit more time, but it works with EF.

It’s actually not a bad alternative since it forces you to focus more on the seeding function, which will also benefit you while writing tests. For prototyping, though, there was nothing as fast as Quino or Atlas [2], both of which, instead of using a metadata table in the database, read the database schema, compared against the application model and applied custom migrations to address differences.

Ferrera finished up with by-now standard advice for adding required columns (and other, similar types of breaking changes): you have to add the column as nullable with a default in the first migration, then get rid of the unwanted default value and nullability in a subsequent migration. You can only remove an unwanted field once the deployed application isn’t using it. That is, you have to drop the column for the deployed version after the version that no longer needs the column. Otherwise, you run the risk of breaking the application that is still running against that database (especially if you have multiple clients/API servers running against the same database instance).

No. No, I’m not.

I was briefly considering it because two good programmer friends [1] of mine asked me, and it seemed like it might be kind of fun to compare our solutions.

But … 24 days, man.

I’ve got other things to do. Like, a lot of other things to do.

I am not in any way bored or looking for things to do.

I’m not even lacking in programming projects that I could be working on.

I’m teaching a JavaScript class right now, for which I’m constantly refining the examples and project code, trimming the sails for the particular people in the current class.

I’ve got a whole web site to which I’ve been meaning to do a bunch of tweaks. I’m just now getting the time to get around to it; I certainly don’t need to prioritize arbitrary programming tasks.

I’ve got a ton of stuff to write that interests me more than writing code for a problem that looks for all the world like a job-interview assessment assignment. I just finished up a 48-page summary of my week’s reading and writing in Links and Notes for November 29th, 2024. I’ve got a stack of book reviews a mile high that I’ve not given up on. Yet. Those kinds of things takes time—and it’s important to me, much more so than the aforementioned programming exercises.

So, no, I’m not doing advent of code. I’d like to compare solutions with my friends, but not enough to reshuffle all of my priorities.

I also wonder who came up with the advent of code? All of the time that people spend every day working on these arbitrary examples is good training, I guess. But training for what? Job interviews? It’s all time that they’re not spending on other creative projects or spending on open-source projects. I’m not trying to knock it at all—becoming a good programmer means just putting in the time.

But the question is what are putting in your time on? When I was the same age as one of my (much younger) friends, I poured all of my time into building a web site—that I’m still using every single day a quarter of a century later.

The featured sponsor for December 8th is Optiver, which says it codes “sub-nanosecond trading systems”, whatever the hell that means. So, they’re trying to find people to make fast trading systems. Just. Stop. We need brilliant coders to be building software useful to society, not to billionaires. Those things are pretty much diametrically opposed. Let’s take a closer look at the list of sponsors.

Apple 1984 Ad

• Jane Street (trading)
• A bunch of online training-course providers…
• Accenture (formerly Anderson Consulting before they had to rename themselves after the Enron debacle, but don’t worry, they’re far, far larger than ever, having failed upward as expected).
• JPMorgan Chase (‘nuff said)
• Lighttricks (AI company…🙄)
• Ahrefs (“indexing” company, but almost certainly a funnel for AI content)
• Shopify (they own the world of online storefronts, having homogenized that part of the world for us; thanks)
• Jump Trading
• American Express
• Bank of America
• ING Bank
• …and a bunch of other, smaller things

It doesn’t look great, if I’m being honest.

Anyway, I’m doing a meta advent of code, in which I just work on as much stuff as I can, but of my own choosing and on my own schedule.

A little while back, someone wrote I can’t wrap my head around MVVM (Reddit), asking for help. I answered with a short example, reproduced below.

tl;dr: Use the MVVM Toolkit and try JetBrains ReSharper or Rider for more IDE assistance for binding and fixing up views. [1]

The concept is that:

• the (M)odel describes your data in the shape you want to store it, process it, etc.
• a (V)iew describes the elements of the UI.
• a (V)iew(M)odel mediates between these two “shapes”.

Why do we need this? Why not just bind the view directly to the model?

Consider a simple person:

record Person(
  string FirstName,
  string LastName,
  Company Company,
  DateTime BirthDate);

The view model might want to expose:

int Age => DateTime.Now.Year − _model.BirthDate.Year;

string FullName => $"{_model.FirstName} {_model.LastName}";

Company Company { get; }

IReadOnlyList<Company> AvailableCompanies { get; }

The AvailableCompanies is for the drop-down menu.

So that’s why there are two models. We don’t want to pollute the data model with view-specific properties. Each view gets its own view model and you can have multiple views/viewModels on the same model. Nice.

The *mechanics* of binding the view to an object has nothing to do with MVVM. It’s *binding*, which is done by magic. This magic is made a lot easier if you use the MVVM Toolkit. The latest versions use source generators so you can actually *see* the magic binding code (in separate source-generated files).

I would also try JetBrains ReSharper or Rider because either of those tools provides a lot more code-completion, hints, warnings, and fixup assistance than a bare Visual Studio does.

I watched this video analyzing a chunk of code, in the hopes of refactoring it.

Always Return Early in Your Code | Code Cop #024 by Nick Chapsas (YouTube)

The original code is the laughably overblown example below.

public List<int> ProcessData(List<int> data)
{
  if (data != null)
  {
    if (data.Count > 0)
    {
      var processedData = new List<int>();
      foreach (var d in data)
      {
        processedData.Add(d * 2);
      }
      return processedData;
    }
    else
    {
      return new List<int>();
    }
  }
  else
  {
    return null;
  }
}

Nick rewrote it as the following:

List<int> ProcessData(List<int>? data)
{
  if (data is not { Count > 0 })
  {
    return []:
  }

  return data.Select(d => d * 2). ToList();
}

Nick’s is OK, but I don’t understand why he bothers to check for Count > 0 when Select() already short-circuits on this case.

@DmitryKandiner rewrote it as the following:

List<int> ProcessData(List<int>? data) => data?.Select(d => d * 2).ToList() ?? [];

This is really short and avoids the unnecessary length-check but it still deals with nullable code, which is silly. There is no need for this function to handle possibly null input data.

I commented the following:

We can also drop the null-check if we have nullability enabled (which any modern project should). Also, I prefer defining APIs with enumerables rather than lists, but if the design insists, I would do it with two methods. This gives callers the option of building lists but doesn’t require them to do so.

List<int> ProcessList(List<int> data)
{
    return ProcessSequence(data).ToList();
}

IEnumerable<int> ProcessSequence(IEnumerable<int> data)
{
    return data.Select(d => d * 2);
}

To which @swozzares replied that I could eliminate the return by using => (called an “expression body”). So I updated the sample with:

List<int> ProcessList(List<int> data)
  => ProcessSequence(data).ToList();

IEnumerable<int> ProcessSequence(IEnumerable<int> data)
  => data.Select(d => d * 2);

And I might as well include the test:

[Test]
public void TestProcessSequence()
{
    List<int> input = [1, 2, 3, 4, 5];
    List<int> expected = [2, 4, 6, 8, 10]; 
    
    Assert.That(ProcessList(input), Is.EqualTo(expected));
    Assert.That(ProcessSequence(input), Is.EqualTo(expected));
}

Someone named @Me_myself_and_I took offense to my comment that projects should be using nullability.

“Except that there are a lot of projects that pre-date that. Its good for new projects sure, but probably not worth the effort to refactor and re-test existing large projects. Devs really need to learn to think about existing systems and long-term maintainability not just new code.”

I think we should be clear about the context. The commentator’s argument is the same one I would make when I work on a legacy project. I think we should take care to show that, if you started a project in C# in the last five years, you shouldn’t be checking for null everywhere anymore. If you turned off nullable checks, then, yeah, you’ll still have to check for it. But you should be designing null-free APIs— where null is not allowed by default.

The article Processes and rules make code review less intimidating by Stefan Judis writes,

“Code reviews are, by nature, intimidating. Sometimes even brutal. If you’ve been in the game for long enough, you probably experienced the following: you worked hard on a feature, you’re proud of yourself and open the PR to be praised and land your changes, and then… it rains comments, suggestions and nitpicks. And if it’s really bad, you’re forced to take multiple feedback and clean-up rounds. It sucks.”

Oh, wow.

Why are people treating writing code like individual school assignments?

That is absolutely not how to effectively use code reviews. That is absolutely not how to work in a team. Teams work together, not against each other.

Why are you putting your heart and soul into your solution? Why is it “your” solution? Is this how y’all were raised? You know, where you prefer being the star genius in your own story, where you managed to get the perfect solution on the first try? Like, you’re the superhero, brilliant, engineer, billionaire playboy?

And then, you learn that you aren’t, and you’re shattered.

But you know what? You’re on a team that’s willing to look at what you made and really try to make it better. Maybe they will make it better! Or maybe they won’t. Both are good! If they do … then it’s better. That’s a win. If they don’t, well, then, you’ve gotten some evidence that supports your theory that what you’ve written really is good and will work.

Up until someone looks at it, you only had a hypothesis that your solution was good enough. You suspected it because you had some code and you had some tests (yes you did, otherwise you have no right to be offended about code-review comments). That’s a hypothesis.

You know who else does reviews to verify theories and hypotheses? Scientists.

Quit your whining. Quit your bullshit.

If you’re treating code reviews like a gladiator arena, as if you were going on Shark Tank or The Voice, then you’re doing it wrong.

The best software is written by a team. It is collaborative. Maybe one person is writing all of the actual text, but there are other minds that contribute advice and feedback that hones the final product.

You know what that sounds like? A writer and one or more editors or proofreaders.

This is how professionals work. Fix your process. Fix your expectations. Fix your fragile ego. Seek validation in a less self-destructive manner.

The author proposes a fix. It’s a technical solution, so it’s not great.

He could have suggested that people do “live reviews” instead of PRs because most people are too lazy or incapable to write critical comments that are also constructive. This is sadly often the case because learning how to write and how to empathize is a lot of work. You could start with empathizing, about which more below.

Instead, he writes about an insipid system where shittily aggressive review comments like “this is not worded correctly” are somehow made better by prepending the text “suggestion:”.

No. It does not make it better.

Why not? Well, for starters, because the text is not formulated as a suggestion. There is only an implicit suggestion that the reviewer would have worded it correctly. This is passive-aggressive time-wasting behavior. On top of that, everything in a code review is a suggestion unless the power dynamic in your team is so severely skewed that we need to be having a different conversation.

The comment should read something like, “I think that something like “… …” would be a clearer way of writing that.”

Or, maybe, you could establish a rapport with the people reviewing your code so you’re not pants-shittingly terrified that you’re going to lose mana when you do one. Maybe you could—gasp!—even be friends. This would help establish a human connection often summarized as empathy wherein the reviewer would consider for a brief moment how a comment might look to the other side and adjust it accordingly, in a manner that is totally not like how robots would do it.

If you’ve established that code reviews are collaboration and not a gladiator arena where “two enter and one leaves”, then the reviewer can be more concise without wasting a lot of time writing curlicue sentences. If you don’t have this rapport, then, yes, I’m afraid you’re going to have to be…what’s the word?…polite.

If you can’t be polite, then, at the very least, you should write review comments that don’t need review comments of their own. You’re going to have to follow the rules of error messages. As detailed in Alerts (Apple HIG),

“Avoid using an alert merely to provide information. People don’t appreciate an interruption from an alert that’s informative, but not actionable.”

Any review comment has to be both informative and actionable. The comment in question—“this is not worded correctly”—is neither. It just vaguely points at something and says “you suck.” It’s clearly attached to a specific line but doesn’t indicate what’s wrong with it. Even if it said specifically what’s wrong, it doesn’t suggest how to fix it.

An error message from a piece of unthinking software can’t go quite that far—unless it’s a spellchecker or grammar-checker!—but an actual, empathetic human in the role of a collaborator can! That person could formulate a suggestion for how to rewrite it so that the review for that line might end after only one cycle. And, as a bonus, it doesn’t end with anyone crying and curled into the fetal position under their desk.

Even if you have an informative and actionable comment, we still come to another downside: they’re still not very efficient. The most efficient way of reviewing code is to do it synchronously or “live”, where the collaborators can discuss and improve the code on the fly, together maybe—and here me out here—even before you’ve even pushed anything! Imagine!

If you’re stuck using PRs and web UIs to communicate, then writing comments like the one in question just wastes everyone’s time. The submitter either will assume what the commentator meant and try again—NOPE, STILL WRONG—or they will have to write “what do you mean?” or “how would you have written it?” This is useless churn. Just write your reformulation with your comment. Remember, you’re a collaborator. You’re not just trying to get through this review as quickly as possible. It’s part of your job.

I was working with a colleague to get the properties that have a particular attribute. The original formulation returned the properties then got the attributes again, plucking the first one off of the list and asserting that it exists to convince the compiler that everything’s OK. We know it exists because otherwise we wouldn’t have returned the property—but the computer doesn’t know that.

Ok, it works but it’s not efficient or elegant. Is there some way to build this so we allocate minimally and don’t have to use the null-forgiving (“dammit”) operator?

I proposed the following formulation. The null-forgiving operator bugs me a bit because I feel like TypeScript would have determined that attribute could no longer be null. C#/Roslyn doesn’t do that.

private static IEnumerable<(PropertyInfo PropertyInfo, TAttribute Attribute)> GetPropertiesAndAttributes<TAttribute>(Type type)
{
    return
        from prop in type.GetProperties()
        let attribute = prop.GetCustomAttributes(typeof(TAttribute), false).FirstOrDefault() as TAttribute
        where attribute != null
        select (prop, attribute!);
}

My collaborator prefers the non-query syntax for Linq, so he rewrote it as follows.

private static IEnumerable<(PropertyInfo PropertyInfo, TAttribute Attribute)> GetPropertiesAndAttributes<TAttribute>(Type type)
{
    return packetType
        .GetProperties()
        .Where(prop => prop.GetCustomAttributes(typeof(TAttribute), false).Length != 0)
        .Select(propInfo => (propInfo, propInfo.GetCustomAttribute<TAttribute>()!));
}

I really don’t like that it calls both GetCustomAttributes() and GetCustomAttribute(), so I looked into how to do emulate let with chained-method syntax. I found Code equivalent to the ‘let’ keyword in chained LINQ extension method calls (StackOverflow) and rewrote the code as follows.

private static IEnumerable<(PropertyInfo PropertyInfo, TAttribute Attribute)> GetPropertiesAndAttributes<TAttribute>(Type type)
{
    return packetType
        .GetProperties()
        .Select(propInfo => (propInfo, attribute: propInfo.GetCustomAttributes(typeof(TAttribute), false).FirstOrDefault() as TAttribute))
        .Where(t => t.attribute != null)
        .Select(t => (t.propInfo, t.attribute!));
}

I still can’t get rid of the second Select() because the type of the first Select() is

(PropertyInfo PropertyInfo, TAttribute? Attribute)

rather than

(PropertyInfo PropertyInfo, TAttribute Attribute)

As in the other formulations, we still need the null-forgiving operator to coerce the type. In the final formulation, it’s much clearer that this is only required for the compiler because the check that attribute is not null is made on the immediately preceding line.

I was curious about TypeScript, though, C# only supports narrowing conversions for inbuilt primitives. Typescript is fancier. I used TypeScript Playground for the examples below.

Here’s where we stand with C#, rewritten in TypeScript:

At this point, TypeScript is making the same complaint as the C# compiler would.

However, if you remove null values from the result, TypeScript recognizes that and automatically narrows the type to number[] from (number | null)[].

I just realized that, while this example is interesting, I hadn’t replicated the example from C# because I wasn’t using tuples.

So, let’s try again and see how far TypeScript gets.

We already have a problem because TypeScript represents tuples with square brackets, which means that this could be an array of two-element tuples or a two-dimensional array. TypeScript defaults to the latter.

We fix that with an explicit type.

If I assign null to b in one element, we have a problem, as expected.

We fix that by adjusting the type of the local variable to be [a: number, b: number | null].

Now, we have the type-conversion error on the result.

We fix that the same way as we do in C#, with a Select, which is called map in TypeScript/JavaScript.

Since TypeScript doesn’t allow you to directly address tuple elements like C# does, we have to “destructure” the elements with const [a, b] = x). So, we have the same thing as in C#, where we allocate “new” tuples for the result. What we don’t have is a ! at the end because TypeScript recognizes the type-narrowing. Clever compiler.

Of course, TypeScript can do this because it’s just transpiling to JavaScript, which plays very fast and loose with types anyway. In C#, the compiler has to make decisions about the shape of the memory it uses, so a Nullable<int> is going to have a different representation than an int. If you want to go from the former to the latter, then you have to define a conversion operator, either an explicit one or an implicit one. Or, as we did, you have to create a new tuple, which entails an allocation. Unfortunate, but unavoidable.

In JavaScript, that’s not the case, at least until it’s run through a JIT, in which case other heuristics would have indicated how to most efficiently shape the storage for that particular instance.

If that sentence doesn’t make sense and you have a follow-up question, then me-from-ten-years-ago is here to offer more in the article Optimizing compilation and execution for dynamic languages, which summarizes a much-longer document written about the WebKit JavaScript engine.

My colleague, who is always game to play with language features, had the good idea to try it with filter().

This confirmed for us that the TypeScript checker doesn’t actually know anything about the effects of the methods filter and map. It’s just that the narrowing happens in the closure passed to the map version, so it can determine that there is no way that b will ever be null when the closure exits. Since it doesn’t know the semantics of filter, it doesn’t know that it actually does that too.

I not unexpectedly very much enjoyed this somewhat-rambling 59-minute course on controlling font features from CSS. It’s really quite amazing and wonderful what you can do declaratively these days.

CSS Typography Crash Course by Kevin Powell (YouTube)

To begin, Roel Nieskens takes a long look at variable fonts, which can be manipulated via both standard CSS properties, like font-weight, as well as using font-variation-settings, all of which can be animated. Variable fonts support a much more granular range of values for font-weight than traditional fonts, all without downloading anything other than the initial font file.

You can use the Wakamaifondue site to determine which features a specific font has, as well as to play with the values along these axes. The standard axes are mapped to CSS properties, like font-stretch, font-style: oblique + angle, font-style: italic, and font-optical-sizing. All of this can also be animated, with the font being able to influence the animation as well.

The demonstrations are quite impressive, especially since it’s all manipulated using a CSS feature that is widely available across major browsers.

CSS is magicalThe next section covers colored fonts, which can contain multiple palettes, each with multiple colors. You can use CSS to override the colors but not directly with color. Instead, you define @font-palette-values to choose a different palette or to override individual colors in a palette. All of this can rely on variables, be animated, and so on, with optimized updates as provided for everything else in a modern web browser.

Next up, he showed how to set font-variant-numeric to tabular-nums to make the font render numbers so that they line up vertically for tabular display. The font has to support this feature but nearly all of them do. This is a good default for table cells. He also shows font-variant-caps and font-variant-numeric to diagonal-fractions, as well as controlling an OpenType feature called “scientific inferiors”, which will subscript numbers, as in chemical formulae, by setting font-feature-settings to “sinf”.

Finally, he talks about standard units like cap (the “the nominal height of capital letters”, according to CSS Length Units (CSS Tricks)). He shows how to do a “true” vertical-align: middle by setting margin-top to calc(1ex − 1cap), which centers without lending so much weight to the descender or ascender.

“Typography is full of details that nobody notices until they’re broken or they’ve gone away.”

I, for one, welcome the high-powered typography features that will let web pages finally look as good as printed output, like magazines and newspapers, where many of these techniques have been used for decades, if not centuries.

While investigating Charts.css, I learned that you can throw unrecognized special characters like square brackets or pipes into CSS class references and its just fine. So you can use them to separate longer lists of classes. For more information, see Cube CSS: grouping by Andy Bell (Piccalilli).

So, you can write:

<article 
class="[ card ] [ section box ] [ bg-base color-primary ]">
</article>

or

<article class="card | section box | bg-base color-primary">
…
</article>

and it works just fine, while being more legible. Charts.css uses it to group related classes:

<table 
class="charts-css 
[ line ] 
[ multiple ] 
[ show-heading ] 
[ show-labels labels-align-start ] 
[ hide-data reverse-data data-spacing-5 ] 
[ show-primary-axis show-data-axes ] ">
  …
</table>

In this otherwise excellent video, I found myself very much wishing that Toub had written at least a single test for the ArrayPool implementation that he built in this video. Still, check out the selected citation below to get a feeling for how they consider performance implications—there are no easy answers, there is only testing and benchmarking.

Deep .NET: Let's Build Our Own ArrayPool with Stephen Toub and Scott Hanselman by dotnet (YouTube)

At 34:45,

“Hanselmann: For folks that may not know what NUMA is: so NUMA is this non-uniform memory access that the computer knows that, like, this CPU is near this memory and…that memory over there, we’re going to consider that remote memory. And it’s all meant to reduce latency. Is that right?

“Toub: Yeah and this definitely factors into things like the GC, right? And even with threadpool scheduling: you want to put the work where the data is.

“Hanselmann: Put the work where the data is. Yes.

“Toub: Otherwise, you spend all your time moving stuff around and thrashing your cache.

“Hanselmann: …and moving things around at different layers of abstraction. Because you would not want to move between NUMA nodes. You don’t want CPU zero to be looking at memory one over there. But then there’s the higher-level question of ‘is the adding of an array pool to my application going to cause memory fragmentation or do I just trust the GC to handle that?’ It’s a constant series of trade-offs. Like, did the complexity I added give me the performance that I wanted or did I just make things more complicated.

“Toub: Well, it’s a great example of where you know I mentioned at the beginning there’s a lot of complexity with pools. And this is a great example of it, If you’re running on a core over here and you use an object that was last used over here, right? Are you better off using that object? Or are you better off just asking the GC to give you a new one? That’s going to be allocated in memory that’s very closely associated with where you are. So these pools aren’t always wins.”

The article Possible Future CSS: Tree-Counting Functions and Random Values by Roman Komarov (Kizu.Dev) is another mathematical master class in using CSS variables and calculations to get at values like “sibling count” and “sibling index”, two values that are in a future proposal for CSS Values and Units Module Level 5 (w3C).

The final demo looks like this, with randomly laid out items squared up into equal columns and rows where possible, all done with only CSS.

Stacking and squaring up grid items

Here’s a taste of the code for getting a random value in CSS,

.random-example {
  & li {
    --random-part-from-sibling:
      pow(var(--sibling-index), 3)
      -
      pow(var(--sibling-index), 2)
      +
      var(--sibling-index);
    --random-part-from-count: var(--children-count);
    --random-limit: var(--closest-prime);
    --random-value: calc(
      mod(
        var(--random-part-from-sibling)
        *
        var(--random-part-from-count)
        *
        var(--seed, 0)
        ,
        var(--random-limit)
      )
      /
      var(--random-limit)
    );
  }
}

You should really see the original article for the interactive demos. As always, it’s stunning how quickly the browser CSS and layout engine efficiently updates values, invalidating only the parts that are affected, even with deeply nested calculations. I went through the article in Opera Beta on an M1 MacBook Pro (from 2020), with a relatively new version of Chromium and it was smooth as silk, with no CPU spikes and no sluggishness (as Komarov indicated might happen in Safari).

He first defines the sibling-count and sibling-index functions, then builds randomness on top of those. He uses this toolkit to build grids that know how many items they have so that he can keep the grid a square with random transforms and coloring. Finally, he even stacks them, with random overlapping and z-order control.

Finally, he links some amazing CSS demos where people built things that could use this functionality in CSS (but have had to make do with JS for now). See Ana Tudor’s many examples or Una Kravets’s radial menu, or Amit Sheen’s demos.

This isn’t a terrible video on IAsyncEnumerable but it’s also not nearly as high-level and fast-paced as I’m come to expect from the .NET Deep Dive series, which is no-muss/no-fuss with Stephen Toub. Those are much better than this one but, if you’re not grokking what IAsyncEnumerable is good for from the documentation or examples, maybe this one-hour video will help. If you’re lucky, it will make you feel better about your own skills as a programmer.

On .NET Live: Supercharge .NET with IAsyncEnumerables: Efficient Data Streams by dotnet / Chase Aucoin (YouTube)

Maybe I’m just super-smart but I can’t understand why so many of Microsoft’s .NET videos spend time discussing the _ separators in numbers. Hanselmann always points it out whenever Toub uses them in a longer constant like e.g., 10_000_000.

Whereas it seems blindingly f”&king obvious what they’re for, Cam Soper in this video just had to ask about them, presumably because, even though he almost certainly knows what they are, he thinks that the audience for a video about IAsyncEnumerables would also be unable to intuit what those symbols might be.

Highlight anything you think seems stupidSo, they get three people involved in a discussion about thousands separators. It’s a waste of time. OMG, I started writing this rant at what I thought was the end of the “basic C# syntax” discussion but I was wrong. They continued for thirty more seconds, with a fourth person chiming in. “C# 7; I just verified.” Congrats, guys! We solved it!

The next thing we got to do was watch the async guru use Task Manager to do memory profiling. Have these guys never heard of Benchmark.Net? Or are they just trying to make other developers feel better about themselves?

I am fascinated that they don’t explain the mechanism behind the IAsyncEnumerable at all. Chase just talks about it as it were magic rather than an enumerable that returns a sequence of Tasks. The magic is in the enumerable part, which allows an algorithm to avoid creating all of the data in memory at once.

The example at 57:00 with System.IO.Pipes, System.Text.Json, and IAsyncEnumerable was quite nice, though. It shows the power of the piping abstraction (which lies below streams). It’s a pity they didn’t get into that more but maybe a follow-up video by Stephen Toub is in the offing.
 

.NET Aspire is a newly introduced tool for building distributed solutions that run just as easily locally as they do in the cloud. This video explains how this is a boon for integration testing.

End-to-end integration testing with .NET Aspire by dotnet / Aaron Powell (YouTube)

The concept is very nice and seems to greatly simplify building integration tests. Kudos and thanks for the introduction.

Still, my hair was standing on end with some of the “fast and loose” programming in this video, though. I know that people will argue that you have to take a direct path to get it working quickly, but I feel that this degrades programming practice, especially when it comes from an “official” source like Microsoft.

There are things like:

• Defining the service-initialization code in the tests, then explaining that it’s to ensure that it’s the “same as that used by the server”. You know how else to do that? Use common initialization code in static helper methods (or whatever).
• Copy/pasting the service-initialization code from test to test
• Copy/pasting the HTTPClient code
• Copy/pasting the record definition, as if that won’t ever bite you in the butt.
• Manually adding “usings” (Can’t you just get the IDE to do that?)

On top of that, the integration tests could have been a lot simpler than they looked if he’d first explained how to set up some common code. Or, perhaps even better, if he’d taken a couple of minutes afterwards to show how to refactor the common code to helper methods (one of which could even be used in the main application so that the app setup is shared with the tests). If he’d used a few more IDE features to speed up coding, he might even have gotten it all in in the same amount of time.

I finally got around to verifying that the defining dependent async methods like the following one is wasteful.

public async Task<bool> N()
{
   return await M();
}

A less-contrived example looks like this:

using System.Threading.Tasks;
public class C {
    public Task<bool> M() 
    {
        return Task.FromResult(false);
    }
    
    public async Task<bool> N() 
    {
        return await M();
    }
    
    public async void RunIt()
    {
        var result = await N();
    }
}

This yields something like the following lowered C# code in SharpLab.IO. Note that there are two state machines.

public Task<bool> M()
{
    return Task.FromResult(false);
}

[AsyncStateMachine(typeof(<N>d__1))]
[DebuggerStepThrough]
public Task<bool> N()
{
    <N>d__1 stateMachine = new <N>d__1();
    stateMachine.<>t__builder = AsyncTaskMethodBuilder<bool>.Create();
    stateMachine.<>4__this = this;
    stateMachine.<>1__state = -1;
    stateMachine.<>t__builder.Start(ref stateMachine);
    return stateMachine.<>t__builder.Task;
}

[AsyncStateMachine(typeof(<RunIt>d__2))]
[DebuggerStepThrough]
public void RunIt()
{
    <RunIt>d__2 stateMachine = new <RunIt>d__2();
    stateMachine.<>t__builder = AsyncVoidMethodBuilder.Create();
    stateMachine.<>4__this = this;
    stateMachine.<>1__state = -1;
    stateMachine.<>t__builder.Start(ref stateMachine);
}

If you write the equivalent code without the await in the N method:

using System.Threading.Tasks;
public class C {
    public Task<bool> M() 
    {
        return Task.FromResult(false);
    }
    
    public Task<bool> N() 
    {
        return M();
    }
    
    public async void RunIt()
    {
        var result = await N();
    }
}

…then you get the following lowered C#. Note that now there is only one state machine.

public Task<bool> M()
{
    return Task.FromResult(false);
}

public Task<bool> N()
{
    return M();
}

[AsyncStateMachine(typeof(<RunIt>d__2))]
[DebuggerStepThrough]
public void RunIt()
{
    <RunIt>d__2 stateMachine = new <RunIt>d__2();
    stateMachine.<>t__builder = AsyncVoidMethodBuilder.Create();
    stateMachine.<>4__this = this;
    stateMachine.<>1__state = -1;
    stateMachine.<>t__builder.Start(ref stateMachine);
}

Manim − linear transformation in 3dThis is a fun video that demonstrates an API, runtime, and IDE called Manim that lets you interactively build 3-D animations. It’s like a game-engine editor [1] in which you build your scenes by calling APIs in Python. There’s an interactive Python terminal, a rendering area, and a text editor.

It’s quite nicely done and he’s put it to good use over the years, building hundreds, if not thousands, of videos with it.

How I animate 3Blue1Brown | A Manim demo with Ben Sparks by 3Blue1Brown (YouTube)

The API is quite high-level and robust but it’s so clear how limited the Python syntax is. He’s very quick with it, but he also knows the whole API by heart. He barely ever used code-completion, so I thought there wasn’t any. But then I saw him hover a few APIs to show the expected parameters. I wonder how much time a novice would spend with interpreter errors. Still, once you’ve gotten used to it, it seems to be pretty efficient. Python’s interpreter speed will never be a problem. In particular, the API for integrating formulae via embedded TeX is pretty neat. It even supports identifying manipulable elements from the rendered version for further animation.

His style of mixing functions and code and variable definitions makes sense for the types of projects he makes. There is going to be very little re-use between these projects. Anything that needs to be reused would eventually be added to Manim itself. He doesn’t seem to see the need for shared libraries. The code is basically throwaway. It takes more time to define common, well-generalized functions than it would to just quickly rewrite it, ready for specialization within that project.

As when watching Kevin Powell ask an LLM about CSS without any idea about how LLMs work, watching Grant Sanderson discuss workarounds for “bugs” without any decent background in languages, scopes, functions, and closures. It’s kind of exasperating watching people “reinvent” computer science without even thinking that there might be prior art—especially clever people.

Good for Grant for making this video, though, because I think he realized that there’s a lot of room to grow in programming skills. He’s already noticed that LLMs aren’t going to help him code in Manim—because he’s an expert and the LLM is definitely not; there is no way it will be able to help him.

The final document of What’s new in C# 13 (Microsoft Learn) is available. There are no major changes for most end users; the changes listed are interested for library and framework developers—especially those interested in writing highly performant code, e.g., Microsoft in its BCL and ASP.NET.

1. Completely unsurprisingly, the params keyword now also applies to IEnumerable<T> (as well as many descendants) as well as Span<T> and ReadOnlySpan<T>.
2. There’s now an official Lock object that, when used instead of the standard object, can lead to more efficient locking code. The .NET runtime and BCL have already starting using this ref struct everywhere.
3. I am not kidding when I say that the third “feature” in the list is that \e is now an accepted escape sequence in all strings. It represents ESCAPE. Um, ok.
4. Method group and method-group calculation has been improved to more closely follow that for overload resolution, allowing Roslyn to better determine a unique type and to extend where var can be safely used.
5. You can now use the “from the end” index operator, ^ in object and collection initializer expressions. Again, this is an improvement that seems like it makes it easier to write input arrays for numeric (data analysis) or tokenizing (LLM) operations.

6. “In C# 13, async methods can declare ref local variables, or local variables of a ref struct type. However, those variables can’t be accessed across an await boundary. Neither can they be accessed across a yield return boundary.

   “[…] You can safely use types like System.ReadOnlySpan<T> […]

   “In the same fashion, C# 13 allows unsafe contexts in iterator methods. However, all yield return and yield break statements must be in safe contexts.”

7. ref structs can now implement interfaces but it’s hard to see the utility because “[t]o ensure ref safety rules, a ref struct type can’t be converted to an interface type.” Um, ok?
8. The language now supports the allows ref struct (anti-)generic constraint. “This enables types such as System.Span<T> and System.ReadOnlySpan<T> to be used with generic algorithms […]” The compiler ensures that the generic code complies with all the rules implied by allowing this more specialized construct.
9. Finally, “you can declare partial properties and partial indexers”. Expanding support for partial enables more flexibility in code produced by source generators.

The work journal 2024-03-27T16:03:51 conversation: 01ht0afgwryks5fepkvvm0kn28 by Simon Willison (GitHub) describes the author’s process of using AI prompting to write a console text-wrapping algorithm.

He prompted with “JavaScript that takes a big string of text and word wraps it at the specified width, adding newlines where necessary.” The answers meandered around a solution space that seemed over-engineered and not particularly fruitful—the answers all used regular expressions, which seems kind of like overkill, when you’re really only trying to detect a single character. A simple parser would be more effective, easier to understand,,debug, and maintain, and is more likely to be performant.

The original wrapping code was:

function wordWrap(text, width) {
  const regex = new RegExp(`(?![^\\n]{1,${width}}$)([^\\n]{1,${width}})\\s`, 'g');
  return text.replace(regex, '$1\n');
}

Things got hairy when Willison asked for a solution that includes an indent before each wrapped line. It’s honestly hard to tell whether the example will actually do that, largely because of the non-obvious regular expressions (which make use of negative lookahead assertions).

function wordWrap(text, width, indent = '') {
  const regex = new RegExp(`(?![^\\n]{1,${width}}$)([^\\n]{1,${width}})\\s`, 'g');
  return text.replace(regex, `$1\n${indent}`).replace(/^/gm, indent);
}

When Willison noted that the indent appears everywhere but at the front of the text, the proposed code was:

function wordWrap(text, width, indent = '') {
  const regex = new RegExp(`(?![^\\n]{1,${width}}$)([^\\n]{1,${width}})\\s`, 'g');
  return indent + text.replace(regex, `$1\n${indent}`);
}

This looks promising, as the eye is drawn to the indent + at the beginning of the answer. If you look carefully, though, you’ll see that the second call to replace() has gone missing.

This is dangerous, right? Imagine if you didn’t have sufficient tests for this code? You can’t really understand it, it looks vaguely right, and pieces that used to work are randomly dropped in responses to “refinement prompts”. He tried a few more times, but kept losing rather than gaining fidelity in the response, with one or more corner-cases no longer being supported.

When he told it to stop using regular expressions, it responded with:

function wordWrap(text, width, indent = '') {
  const lines = [];
  let currentLine = '';

  text.split('\n').forEach(line => {
    if (line === '') {
      lines.push('');
    } else {
      line.split(' ').forEach(word => {
        if (currentLine.length + word.length <= width) {
          currentLine += (currentLine ? ' ' : '') + word;
        } else {
          lines.push(indent + currentLine);
          currentLine = word;
        }
      });
      lines.push(indent + currentLine);
      currentLine = '';
    }
  });

  return lines.join('\n');
}

This actually seems better—and Willison claims it works—but it still seems only OK. There is a lot of allocation in this response, as well as a failure to separate concerns.

There are a few pieces to this request:

1. Split the string into paragraphs
2. Wrap each paragraph to lines of a given width, including an optional prefix on each line

You can play with his version at Wrap text at specified width by Simon Willison (Observable)

When I threw the code into WebStorm and added a test, I discovered that it doesn’t wrap at the desired width.

• It fails to take the length of the indent into account when wrapping the text.
• It doesn’t sanity-check that the indent isn’t bigger than the desired wrap length.

At least the algorithm doesn’t fall into the pathological trap when a word is too long on it’s own to fit within the desired width, which would result in an infinite loop.

I noodled around with a solution that uses generators to avoid allocation until you actually snip text. The algorithm looks like this:

• Pass in a text, desired width, and indent.
• Get a generator for all line breaks.
• Get a generator for all word breaks in a line.
• Yield a generator for all line spans.
• Yield a generator from that for all lines.

This strategy ensures that there are no substring allocations until you actually need them. You can get the first line or two lines without allocating more than the substrings for those lines. There are no substrings allocated to find word breaks, as the algorithm outlined above does.

Is it a more complex algorithm? Of course. Is it relatively easy to understand, especially with the requisite tests? Yes. Does it do its job much more efficiently? Absolutely.

The article Prefer test-doubles over mocking frameworks by Steve Dunn writes,

“This is testing implementation and not behaviour. Your SUT called something and there is likely an observable side-effect of that. Test the side-effect and not that a particular method was called. If the code is refactored (e.g. you change the implementation but not the behaviour), then your test that checked that a method was called will likely break, but your test that tested the behaviour should remain unchanged and should still pass.”

I think we have to be more careful here. Sometimes you want to test the implementation, no? If you look at the simplest test double that he’s written in the article, shown below, you can see that there is an implicit assumption that would have to be tested: that is, that the Get method in the test-double accurately represents the actual implementation.

This is the interface to be tested.

public interface IProductRepository
{
    void Store(Product product);
    Product Get(int id);
}

This is the test using the test double:

[Fact]
public void Using_test_doubles()
{
    var repo = new InMemoryProductRepository();

    var sut = new ProductService(repo);

    sut.OnboardNewProduct(123, "Product 123");

    repo.DidStore(123).Should().BeTrue();
}

Note that the test calls a test-double-only method called DidStore(), which is assumed to have been implemented as expected. A naive implementation would just return true. Since this is a test double, there are no tests verifying that it doesn’t always return true. Shouldn’t the test instead verify that the product is not stored first—i.e., repo.Get(123) returns false—before calling OnboardNewProduct(123, …) and then testing repo.Get(123) again to verify that it returns true?

The following is the implementation of the test-double.

public class InMemoryProductRepository : IProductRepository
{
    private readonly List<Product> _products = new();

    public void Store(Product product) => _products.Add(product);
    public Product Get(int id) =>  _products.FirstOrDefault(p => p.Id == id);

    // This is not part of the interface, but is useful for testing
    public bool DidStore(int id) => Get(id) is not null;
}

If you leave the test as formulated, there is literally no guarantee that anything changed at all. The author is simply assuming that Store adds a product because he can see that it does.

The author wasn’t quite clear why his mock-based implementation isn’t good, though. He proposed the code below.

[Fact]
public void Using_mocks()
{
    var repo = Substitute.For<IProductRepository>();
    var sut = new ProductService(mock);

    sut.OnboardNewProduct(123, "Product 123");

    repo.Received().Store(Arg.Is<Product>(p => p.Id == 123));
}

Do you see how he checked whether the Store() method had been called rather than testing whether Get(123) returns true? He had to do that because the mock would always return false unless the author had also set up the Get() method to return true if the method were to be called with 123. Why wouldn’t he do this? Because he’d then have just been testing the mock. However, if you look closely at the previous example, the author is also just testing his test-double.

I have another problem with the statement above: sometimes I very much want to verify that a specific method is being called. I’m not trying to verify the behavior of the test-double; I’m trying to verify the behavior of the actual implementation.

If, for whatever reason, I can’t use the actual implementation, then I want to verify that a certain method was called because e.g., I know that that method calls a system API directly. That is, I trust that the system API will do what it says on the tin. I’m able to verify manually that the parameters to the method are passed on to the API faithfully. I can’t call the API in the test suite—maybe it’s a call to the Windows Registry or maybe it’s accessing a USB stick that doesn’t exist in CI—but I can get as close as possible. If something still goes wrong, then I know that I just have to examine the one line of code in the actual implementation. In that way, I’ve verified a fact about the system that means something.

This comes up often enough in more complex component graphs, where you’ve had a bug that, under certain circumstances, a certain notification is not sent. In that case, you might be unable to verify that the message arrives—as we do by testing Get(123) above—because the actual message would go through an online proxy like Apple and would end up on a mobile device somewhere, and maybe you don’t want to build the testing infrastructure that mocks a receiving device that you can check. It wouldn’t help you because you’d just be testing the test-double implementation anyway.

Instead, you would trigger a high-level API that, eventually, bubbles through several layers until the notifier is triggered with a certain message. In that case, an efficient and effective test would be to test that the INotifier.Send() method was called with the expected parameters.

Even in the author’s example, there is presumably an external data store of some sort that is being mocked. I’m not actually interested in testing whether that data store interprets my command to store correctly. I’m going to assume that it does because it’s not my code.. [1] What I want to confirm is that I sent the command to the store. That is, I want to verify that a particular method was called with particular parameters. Perhaps I’ll use a snapshot test to verify that the generated SQL is correct. Then I don’t have to actually run the SQL against the database every time.

In the author’s case, he’s calling a method on one interface and verifying that a property of another interface has changed. He is testing the interplay of those two components. That he used test-double doesn’t help at all—it’s because the test-double was written correctly that the test means anything. And there are no tests to verify that the test-double actually does what he assumes it is doing.

While I agree that test-doubles have their place, I think that mocking frameworks can also be very helpful. That’s why I don’t like rules like “test behavior not implementation”. I prefer to consider it a guideline, so that I can remember to write high-level, well-abstracted tests where possible but I can also just test that a certain method on a certain component will be executed.

The Talk − Bringing C# nullability into existing code by Maarten Balliauw is a 66-slide deck that I summarize as follows:

• The C# nullability feature is for build- and design-time. It does not enforce anything at runtime. That means that you still have to check parameters for null.
• The C# nullability feature is available to solutions working with .NET Framework and .NET.
• For .NET Framework, you have to explicitly set the <LanguageVersion> to 8.0 (however, there are a bunch of cons associated with doing this, as the runtime library itself is not annotated). [1]
• The presentation shows how to enable and disable for the whole solution, project, or an individual code region.
• For new solutions, enable at the solution level.
• For small solutions, enable at the solution level and just work through it.
• For large solutions, enable project-by-project or file-by-file—or even class-by-class.

  • “Start at the center and work outwards.”
  • While ? suffices in most cases, consider annotations to improve your own APIs
  • Consider redesigning APIs that return null (use the bool TryGet<T>(…, out T) pattern or return a “null” object instead).
  • Avoid allowing null parameters (these force a decision on the implementation that is often better handled by the caller).
  • Don’t use ! except temporarily
  • Don’t use suppression except temporarily
  • Start with types that aren’t depended on a lot. Those are easy.
  • Take types with lots of dependents one-by-one.

Introduction

This article defines concepts like repository and working tree and then discusses how you can use .gitignore files to determine the files and folders that Git considers during operations.

Concepts

From a command line, you can run git init in any folder to make any folder a Git repository. Doing so creates a .git folder with a database and configuration files for the local repository. Git considers any folder that contains a .git folder with these configuration and database files in it to be a Git repository.

The working tree of such a folder is all files and subfolders in that folder other than the .git folder.

Any non-trivial repository will contain at least some files that should not be committed to the Git repository. Commonly, these are build artifacts produced from the source files that are included in the repository.

.gitignore

A repository may include multiple .gitignore files. From the official documentation]:

“Patterns read from a .gitignore file in the same directory as the path, or in any parent directory (up to the top-level of the working tree), with patterns in the higher level files being overridden by those in lower level files down to the directory containing the file. These patterns match relative to the location of the .gitignore file.”

Git determines whether a given file is included in the working tree by collecting the patterns from any .gitignore file anywhere in that file’s path. Patterns defined “closer” to the file override those defined “farther” away.

For example,

[Root]
  📄 .gitignore
  📁 ProductMedia
     📁 Win10
        📁 Content
           📄 .gitignore
           📁 Deploy
              📁 Control
                 📄 .gitignore
                 📄 readme.txt

Git determines whether readme.txt is included in the working tree by collecting the rules from files in the following order:

• .gitignore
• ProductMedia/Win10/Content/.gitignore
• ProductMedia/Win10/Content/Deploy/Control/.gitignore

Patterns

A .gitignore file contains zero or more patterns.

• Each pattern excludes the affected files from the working tree
• Each pattern affects only files in the sub-tree defined by the sub-folder in which the pattern is declared.
• Each pattern is interpreted relative to its containing folder (i.e., the root folder for a pattern is its declaration folder).

The following examples illustrate common patterns.

The column “Disables subsequent patterns” indicates whether subsequent patterns that target files in the ignored sub-tree will have an effect. As detailed below, the presence of a trailing * determines whether Git considers subsequent rules. This only becomes relevant for re-including deeply nested files.

obj/

All files in any folder named obj anywhere in the sub-tree of the current folder or any sub-folder thereof.

Disables subsequent pattern: Yes

/obj/

All files in the root folder named obj or any sub-folder thereof.

Disables subsequent pattern: Yes

/obj/*

All files in the root folder named obj or any sub-folder thereof.

Disables subsequent pattern: No

*.bin

All files ending in .bin anywhere in the sub-tree of the current folder.

Disables subsequent pattern: Yes, but irrelevant

src/*/out/bin

All files in any path named out/bin found in any single sub-folder of any folder named src anywhere in the sub-tree of the current folder.

Disables subsequent pattern: Yes

src/**/out/bin

All files in any path named out/bin found anywhere in any combination of sub-folders of the sub-tree of any folder named src anywhere in the sub-tree of the current folder.

Disables subsequent pattern: Yes

Re-including files

You can also re-include files with the ! operator. For example, you might want to exclude everything in a folder but a single configuration file.

The following example ignores everything in the /out folder except for the file settings.json.

/out
!/out/settings.json

Difference between / and /*

As very nicely explained in the answer on StackOverflow .gitignore exclude folder but include specific subfolder,

• The pattern dir/ excludes a directory named dir and (implicitly) everything under it. With dir/, Git will never look at anything under dir, and thus will never apply any of the “un-exclude” patterns to anything under dir.
• The pattern dir/* says nothing about dir itself; it just excludes everything under dir. With dir/*, Git will process the direct contents of dir, giving other patterns a chance to “un-exclude” some bit of the content (!dir/sub/).

For example, the following patterns ignore all files in obj and bin folders but files named readme.md in those folders should be included. The following patterns achieve this for obj but not for bin folders (because the missing * prevents Git from considering the rule re-including readme.md for bin folders).

/obj/*
!/obj/Readme.md
/bin/
!/bin/Readme.md

This distinction is highly relevant for the next section.

Re-including deeply nested files

Suppose we have a folder structure as shown below.

[Root]
  📁 Folder1
  📁 Folder2
  📁 Folder3
  📁 ProductMedia
      📁 Win10
         📁 Content
           📄 Other files…
           📁 Deploy
             📁 Control
               📄 CustomerSettings.ini
               📄 Bootstrap.ini
               📄 Settings.xml
               📄 Other files…
           📁 Folder1
           📁 Folder2
           📁 Folder3
           📁 Other folders…
      📁 Win11
         📁 Content
           📄 Other files…
           📁 Deploy
             📁 Control
               📄 CustomerSettings.ini
               📄 Bootstrap.ini
               📄 Settings.xml
               📄 Other files…
             📁 Folder1
             📁 Folder2
             📁 Folder3
             📁 Other folders…
           📁 Folder1
           📁 Folder2
           📁 Folder3
           📁 Other folders…

We want the patterns that will ignore everything in any files in the Content folder of any sub-folder of ProductMedia except for the CustomerSettings.ini, Bootstrap.ini, and Settings.xml files under the path Content/Deploy/Control/ of any sub-folder of ProductMedia/.

You might think that the following would do the trick:

# Ignore everything in the "Content" folder of any subfolder of "ProductMedia"
ProductMedia/*/Content/*

# Except for the specified files under the path "Content/Deploy/Control/"
!ProductMedia/*/Content/Deploy/Control/CustomerSettings.ini
!ProductMedia/*/Content/Deploy/Control/Bootstrap.ini
!ProductMedia/*/Content/Deploy/Control/Settings.xml

Microsoft Copilot certainly thought so. This does not have the intended effect.

The pattern correctly uses the * to indicate that Git should continue processing patterns for sub-trees of the excluded folder. It also correctly ensures that the rule applies both to Win10 and Win11 folders by using a *.

However, while patterns affecting the path Content will be considered, those affecting Content/Deploy or any sub-folder thereof will not.

The trick, as outlined in the StackOverflow answer, is to re-include, then exclude each individual sub-folder in the path, as shown below.

# We ignore the generated "Content" subfolders in the ProductMedia folder
ProductMedia/**/Content/*                    # Allow subsequent processing
# Except for certain configuration files
!ProductMedia/**/Content/Deploy/             # Include "Deploy", allowing subsequent processing
ProductMedia/**/Content/Deploy/*             # Ignore everything in "Deploy"

!ProductMedia/**/Content/Deploy/Control/     # Include "Content", allowing subsequent processing
ProductMedia/**/Content/Deploy/Control/*     # Ignore everything in "Content"

# Finally, include the desired individual files

!ProductMedia/**/Content/Deploy/Control/CustomSettings.ini
!ProductMedia/**/Content/Deploy/Control/Bootstrap.ini
!ProductMedia/**/Content/Deploy/Control/Settings.xml

This pattern of re-including, then re-excluding each sub-folder suffices to allow Git to consider the file patterns at the end, while still ignoring all other files in any sub-trees that would otherwise have been included.

This is another excellent 1-hour tour of another complex corner of .NET. Toub describes and shows how the source-generated RegEx engine works.

Deep Dive into RegEx with Stephen Toub by dotnet / Scott Hanselmann (YouTube)

• The generated source is human-readable and debuggable.
• It is well-commented.
• It updates in real-time as you change the expression.
• It includes XML documentation that describes the regular expression in plain English.
• They rewrote the compiler in .NET 7 to not only better support source generators, but also to be able to emit not only IL, but source code. They rebuilt the emitter to allow more leeway in code-generation—the first generation emitted C# that looked very much like IL.
• They have a gigantic test-suite that they culled from open-source code. 4M expressions deduplicated down to about 20,000 unique expressions that they have in the test suite and that they run against all four RegEx engines to verify that nothing runs pathologically long or with excessive memory.
• There is an analyzer that tries very hard to eliminate greediness. It seeks atomicity. Fascinating.
• At 47:00, he shows a great example of a regex that requires backtracking, which can lead to pathological, exponential performance. These engines support back-references, which are powerful. They can be super-fast for matches, but they have very bad worst-case behavior that may end up in DDOS behavior. In .NET, you can set a timeout on your regular-expression evaluation to avoid this. You can also set a global timeout. You can also turn off back-tracking. If it can produce the engine to evaluate the expression, then it will evaluate in linear time. If it cannot, it’s probably a compile-time error if you’re using source generators, which is quite nice.
• They also examine an email-address RegEx, which takes Toub into showing how the generated source uses the SearchValues variants, which are a highly-optimized way of searching text, with dozens of algorithms that it chooses by analyzing the input string. They have SIMD/Vector/Arm Intrinsics support where possible and are exactly the kind of optimization that a framework like .NET can offer, but that an app developer would never have time to make.

💙 Stephen Toub. He’s absolutely brilliant. Mad props to Scott Hanselmann for reining him in and providing a great sparring partner.

This is another video from Stephen Toub that is just chock/full of useful information.

Writing async/await from scratch in C# with Stephen Toub by dotnet (YouTube)

At 27:30, they start to discuss about the nomenclature of Task and how it differs from an Action. It’s funny that neither of them mentioned that tasks in .NET are called promises pretty much everywhere else (JavaScript, Java, etc.). Some libraries also use the word future. For more information, see Futures and promises (Wikipedia).

As he’s building everything, it is really astonishing to note that Hanselmann has to tell Toub that you can have Visual Studio generate methods for you. How does he not know that? When he did it, he then used the mouse to select “Find References” from the shortcut menu instead of just pressing F12. When he got to the method, he said “Oh, it didn’t implement it,” as if disappointed that Copilot hadn’t botshit a version in there for him. He was going to write it himself anyway, but it was telling that he’s gotten so accustomed to Copilot just filling in implementation.

A little while later, he’s learned the new tool, telling Hanselmann that he’s going to use his “trick” to create the method.

At around 52:30, he implemented a try/catch to “be a good citizen” and accepted what Copilot had recommended for him, but it didn’t match what he said he was writing. He said “so we always set the task result” but the code that he/Copilot wrote returned from the catch, which means that the task result isn’t going to be set when there is an exception. Now I don’t know which one he meant: what he said he wanted to write (did he misspeak?) or what he actually wrote (which Copilot wrote for him and he might have automatically accepted).

Since he has no tests whatsoever, this is exactly the kind of subtle bug that might go undetected for quite a while, as it’s in the exception-handling code. It might also be quite difficult to diagnose.

When he wrote the exact same thing again at 1:00:00, he seemed to indicate that what it wrote was OK: i.e., it either sets the exception or it sets the result.

At 56:00, he gets to the point of trying to get to the synchronous calling style supported by await and builds his own logic. It works, but it still can’t be used with the await keyword. He quickly implements a TaskAwaiter and voila! 🧙‍♂️ it works! His very own implementation of the Task pattern that integrates with the compiler.
 

This is a great interview with the master of performance-optimization in .NET Stephen Toub. If you’re relatively well-versed in C#, .NET, and Linq, then you can just jump to the second video (linked below). I actually watched the second one first. I didn’t feel like I’d missed anything.

Deep Dive on LINQ with Stephen Toub by dotnet (YouTube)

Stephen Toub’s the guy who writes the 100+-page release notes on performance. See the following links.

• Performance Improvements in .NET 5 (46 pages)
• Performance Improvements in .NET 6 (109 pages)
• Performance Improvements in .NET 7 (170 pages)
• Performance Improvements in .NET 8 (235 pages)
• …more to come.

In this video, at 26:00, Scott flubbed the joke. It doesn’t really matter but, according to the article Two Hard Things by Martin Fowler, the original saying was:

This was “upgraded” to:

At 33:00, Toub talks about how the Current property of IEnumerable is not checked for nullability because … it can technically always be null, but we also don’t use it that much and we don’t want the compiler yelling at you for possibly-null access when the item type is a value type (for example) or a non-nullable reference type. See the code for the annotation and more comments.

At 49:00, they discuss the use of goto, where Hanselmann says, “there’s the kind that you can see; and there are the kind that are hidden,” which I feel like he fumbled as well. I think what he meant was that there are implicit gotos everywhere. A goto maps to an assembler jmp, so every if has an implicit goto. The break statement inside of a case block of a switch statement jumps (or goes to) the end of the switch block.

You should be careful about using goto—i.e., it’s a code smell—but sometimes it’s the clearest and most concise way of expressing intent. In their case, they used it to “fall through” from one case statement to the next with goto case 2.

I feel the same way about goto as I do about the continue keyword in a loop. I like to write all of my loops in a consistent and idiomatic manner. That means I use break. So, for me, using continue is a code smell, but sometimes it’s the most elegant way of expressing the intent.

At about 55:00, Hanselmann shows Toub how to use Winget to install SysInternals and then to use ZoomIt for presentations. This was a nice bonus that probably a lot of viewers didn’t know: (A) Zoomit is a poor tool compared to Ctrl + mouse-scroll on MacOS, but it’s better than nothing and (B) running Winget from PowerShell is a pretty powerful install-and-update tool on Windows.

In this presentation, I notice that Toub doesn’t use the “extract variable” refactoring either. I’m not sure whether something like that is available in VS. He also installs packages with the VS NuGet interface rather than just adding the package reference manually in the project file. I note other inefficiencies in his coding style in the notes to the second video below. He’s fast! …but he could be even faster.

An even DEEPER Dive into LINQ with Stephen Toub by dotnet (YouTube)

At about 22:00, he mentions something interesting that makes me change my opinion about the type to use for private variables. I’d always used interfaces to keep it clear which API my implementations depended on. This included list variables, which I would type as IList<IBase> instead of List<IBase>. Toub says that the second formulation is better for performance because the compiler doesn’t have to deal with virtual dispatch—it can just call the functions of List directly.

This makes me realize that it’s a good reason to use var. When you use var with new, the resulting type of the local variable will automatically be the “right” type for performance. The same goes for target-typed new expressions, which force you to use a non-interface type for fields.

At 27:00, they talk about the IDE features for tracking change and inspection states—the colorful splotches of color in the left and right gutters and on the vertical scroll bar. First of all, Toub doesn’t know left from right, but he’s a genius so we’ll grant him that. Second of all, neither he nor Hanselmann really understands what the file-state markings are.

The IDE tracks not only unsaved changes but also uncommitted changes. That’s why there is so much green in the gutter on the left and the right: because Toub had rewritten much of the file since the last commit but he’s been saving the file the entire time. It’s an important distinction to make for understanding what’s going on. The right-hand side of the right-hand gutter shows inspections: suggestions, warnings, and errors.

It was interesting to see that, since Toub uses Visual Studio without ReSharper, he instinctively used the mouse to copy/paste the name of the constructor from the class after he’d renamed the class. That is, he’d renamed the class and, in order to fix the constructor being named incorrectly, he copy/pasted with the mouse. With ReSharper, he could have just pressed Alt + Enter and selected the quick-fix called “This is a constructor”, which just renames the method to the name of the containing class.

I could see several places where he used the mouse rather than being able to stay on the keyboard, letting the IDE do his work for him. For example, he also copy/pasted the name of the iterator class again in order to use the more highly specialized version—but he could have just started typing to have the auto-complete suggest the right type. Or he could have used multiple clipboards to paste the type name that he’d just copied before (when he’d fixed the constructor).

When the type of the actual argument no longer matched the formal argument, he copy/pasted again to get the more specific type. Here, he could have also asked ReSharper to show variables in scope with an appropriate type. It would have shown only array and been done with it. No mouse, no copy/pasting, no moving away from the keyboard, no guesswork.

At about 35:00 or so, shit gets real, as Toub starts hand-optimizing the code for his iterator. He does some obvious stuff first, removing iterator complexity that is no longer needed when the iterator has to handle only fixed-length and integer-indexable arrays. Next, though, he does a neat trick with a uint cast that ensures that a check for i < source.Length will never be true, even if i is less than 0 (because (uint)-1 wraps to approximately 2 × int.Max, so it will never be less than Length. After that, he talks about how the jitter can use that condition to avoid the automated bounds-checking that comes with .NET’s managed code by default when indexing the array.

So the cast to uint not only avoids a branch, it also avoids the hidden cost of bounds-checking. Nice! 👏 After running the benchmark again, we can see that he’s rebuilt the optimizations available in C# by default. Very nice! 👏 👏 This is why running against a newer runtime and library may increase the performance ⏱ of your own code. Toub mentions that these types of optimizations are great, but they have to balance the value of the optimization versus the additional code to maintain as well as the size of the runtime assemblies.

At 49:00, he mentions that the size issue also affects Native AOT, since AOT can’t take advantage of PGO or anything else that the jitter has available to eliminate unnecessary code. AOT doesn’t have as much information available, so it can’t optimize away as much code. That is, if it can’t guarantee that only the array-optimized version is called, then it has to keep all of the versions, increasing the binary size. There are also vector-optimizations and SIMD-optimizations that may have to be included. For more information, see AOT, JIT, and PGO in .NET.

The extremely detailed chapter overview.

00:00:00 Deep Dive into Implementing Iteration in Programming
00:01:50 Understanding the Implementation and Functionality of Custom Iterators in Link
00:07:45 Discussing Optimization Strategies and Array Specialization in Programming
00:10:43 Understanding the Use of Sharp Lab and Compiler Optimization in C
00:15:20 Discussing Optimizations in Link Methods in Programming
00:16:39 Understanding SIMD and its Application in Computer Processing
00:20:12 Discussion on Code Analysis and Optimization Techniques in Software Development
00:23:46 Discussing and Implementing Iteration-Based and Manual Arrays in Programming
00:30:30 Exploring Compiler Optimizations and State Management in Programming
00:37:04 Exploring Hyper and Micro Optimization in Programming
00:40:11 Exploring Code Optimizations and Trade-offs in Programming
00:45:41 Discussing the Challenges and Implications of Optimizations in Software Performance
00:47:54 Discussing the Implementation and Optimization of Select in Programming
00:51:49 Discussion on Programming Syntax and Benchmarks
00:54:41 Implementing and Discussing Iteration Code in Programming
00:57:12 Understanding the Functionality and Implementation of C# Compiler Keywords
01:02:42 Improving Functionality and Performance of Manual Implementation of Iteration Methods
01:08:39 Exploring the Optimization and Implementation of Select Operators in Programming
01:12:57 Understanding and Optimizing Iteration Operations in Programming
01:18:09 Implementing and Utilizing LINQ Programming: A Two-Parter

This video is from a great channel, which published a lot of great videos a while back. They covered pretty much everything already, but circled back to JavaScript for 2024.

Interview with Senior JS Developer 2024 by Programmers are also human (YouTube)

Some choice quotes from the video.

“We push on save.”

“2024 is the year of the serverlesslessness.”

“They say that every year, but this year they’re out of VC funding.”

“Don’t write this down, next week all of this is gonna change.”

This guy just keeps knocking it out of the park. Pretty much everything he mentioned exists and is as described.

The following video discusses the downsides of the current implementation of primary constructors:

The C# 12 Feature You Shouldn’t Use, Yet by Nick Chapsas (YouTube)

To sum up:

• Primary constructors don’t have a readonly backing field; you can still assign to it within the type.
• You can’t control the visibility of the generated property or backing field.
• You can’t throw exceptions, except in a field-initializer, which isn’t as obvious or clean as doing so from within a standard constructor

Nick contrasts the C# implementation with the language feature in Kotlin, which allows all modifiers in the declaration, but has the same problem that the class definition can get pretty wordy.

The article Primary Constructors – Using C# 12 in Rider and ReSharper by Matthias Koch (JetBrains Blog) describes another ugly phenomenon: double capture.

Let’s consider the following example:

public class Person(int age)
{
    // initialization
    public int Age { get; set; } = age;
    // capture
    public string Bio => $"My age is {age}!";
}

In this class, the parameter age is exposed both through the Age and Bio property. As a result, the object stores the state of age twice! For reference types, a double capture leads to an increased memory footprint and possibly even memory leaks. In our concrete example, you will observe the following unintended behavior:

var p = new Person(42);
p.Age.Dump();   // Output: 42
p.Bio.Dump();   // Output: My age is 42!
p.Age++;
p.Age.Dump();   // Output: 43
p.Bio.Dump();   // Output: My age is 42! // !!!!

The article Continuous Integration by Martin Fowler makes many interesting points. It is a compendium of know-how about CI by one of the industry heavyweights, who’s been using it for a long time.

While I found a lot of what he had to say interesting, I did wonder how applicable CI is for the kinds of teams that I know and work with. He makes several statements toward that end that pretty severely limit the applicability of what he calls “true CI” for many, if not most, teams.

I think he should have started his article with a very clear delineation for which kinds of organizations this kind of process is appropriate or efficient. In leaving it out, he seems to suggest that it’s the best for everyone, but at the end of the article, he lists what are, for me, quite severe restrictions. For example,

• “Continuous Integration is more suited for team working full-time on a product […]”
• “[…] elite teams deployed to production more rapidly, more frequently, and had a dramatically lower incidence of failure when they made these changes.”
• “If a team attempts Continuous Integration without a strong test suite, they will run into all sorts of trouble”

I don’t get the impression that Fowler is discussing a dream scenario toward which one works, but rather what he considers to be the absolute minimum process that anyone should be utterly embarrassed about themselves for not already having. I didn’t see a single sentence in this 40-page, at-times repetitive document about how to actually get there from here—or whether that’s really appropriate for many projects that people who read Martin Fowler might be working on.

I wonder about the wisdom of prioritizing integration seemingly above all else.

Below are citations from the long paper, with my comments interleaved.

“This contrast isn’t the result of an expensive and complex tool. The essence of it lies in the simple practice of everyone on the team integrating frequently, at least daily, against a controlled source code repository. This practice is called “Continuous Integration” (or it’s called “Trunk-Based Development”).”

He says this a lot, but I never hear about the costs. Is there no amount of time lost on integrations that is too high a price? Is there no task that he doesn’t break down into a million pieces in order to accommodate this style of work? Is there no efficiency lost by making each task into 1-hour chunks of coding that the entire team then integrates? Is that what we’re doing now?

“This will consist of both altering the product code, and also adding or changing some of the automated tests. During that time I run the automated build and tests frequently. After an hour or so I have the moon logic incorporated and tests updated.”

I’m quite fed up with reading this kind of optimistic bulls%!t. What kind of programmers are these who can accomplish major work in one hour? Or are the tasks that Fowler can conceive of all so simple that they can be accomplished in an hour? I’m very suspicious about these kinds of statements. It reminds me of game developers in the 90s talking about how they’d “written the whole engine in a weekend”, but then the game still took five more years to deliver.

“Some people do keep the build products in source control, but I consider that to be a smell − an indication of a deeper problem, usually an inability to reliably recreate builds. It can be useful to cache build products, but they should always be treated as disposable, and it’s usually good to then ensure they are removed promptly so that people don’t rely on them when they shouldn’t.”

Sure. But—priorities. Your product is not the pipeline. It’s your product. You can’t make everything a slave to the process. Remember to fix that which you can fix quickly, but to focus on your own priorities. Don’t polish a build so that Martin Fowler is happy, if it’s going to make your customers wait a lot longer for their release.

“The tests act as an automated check of the health of the code base, and while tests are the key element of such an automated verification of the code, many programming environments provide additional verification tools. Linters can detect poor programming practices, and ensure code follows a team’s preferred formatting style, vulnerability scanners can find security weaknesses. Teams should evaluate these tools to include them in the verification process.”

“Everyone Pushes Commits To the Mainline Every Day

“No code sits unintegrated for more than a couple of hours.”

This feels completely divorced from reality, but maybe I just “don’t get it.”

“If everyone pushes to the mainline frequently, developers quickly find out if there’s a conflict between two developers. The key to fixing problems quickly is finding them quickly. With developers committing every few hours a conflict can be detected within a few hours of it occurring, at that point not much has happened and it’s easy to resolve. Conflicts that stay undetected for weeks can be very hard to resolve.”

I agree with the last sentence, but at what cost? It feels like you’re going to spend so much time committing and integrating. How is finding out if you have conflicts the highest-priority task your team has?

“Full mainline integration requires that developers push their work back into the mainline. If they don’t do that, then other team members can’t see their work and check for any conflicts.”

Who finishes anything non-trivial in an hour? I can’t escape the feeling that one-hour chunks is almost too granular, that this size was chosen because it aids integration. While that’s a noble goal, I wonder how appropriate it is for many tasks, and to what degree the shape of the process affects the size of the solution set.

“Since there’s only a few hours of changes between commits, there’s only so many places where the problem could be hiding. Furthermore since not much has changed we can use Diff Debugging to help us find the bug.”

But don’t you waste time hunting bugs that would have gone away by themselves if the process weren’t so frenetic? If you rebase everything, then you’ll still encounter every integration conflict. If you merge, though, you can skip many of those interim integrations because subsequent changes might have obviated prior ones that might have caused conflicts.

Instead of testing occasional version, you end up testing absolutely everything you do as if it were a release candidate. I’m not convinced that there’s no downside to that. I feel like it’s a waste of time if applied so mindlessly.

“Often people initially feel they can’t do something meaningful in just a few hours, but we’ve found that mentoring and practice helps us learn.”

I don’t know who you’re working with, but I wonder how useful is that? How useful is it to tailor your entire process to ruthlessly chopping up your work into tiny segments? What if that’s not how some people work? What if they can’t learn? Fire ‘em?

“Continuous Integration can only work if the mainline is kept in a healthy state. Should the integration build fail, then it needs to be fixed right away. As Kent Beck puts it: “nobody has a higher priority task than fixing the build”.”

You goal ends up being running to run the process, rather than to build the product. This sounds more and more like a cult.

“If the secondary build detects a bug, that’s a sign that the commit build could do with another test. As much as possible we want to ensure that any later-stage failure leads to new tests in the commit build that would have caught the bug, so the bug stays fixed in the commit build.”

“A team should thus automatically check for new versions of dependencies and integrate them into the build, essentially as if they were another team member. This should be done frequently, usually at least daily, depending on the rate of change of the dependencies.”

This seems like another thing that becomes a higher priority than building the product itself. Daily dependency check seems like overkill, but it’s automated, so who cares? He’s just running builds all the time, like we don’t have a climate crisis.

“if we rename a database field, we first create a new field with the new name, then write to both old and new fields, then copy data from the exisitng old fields, then read from the new field, and only then remove the old field. We can reverse any of these steps, which would not be possible if we made such a change all at once. Teams using Continuous Integration often look to break up changes in this way, keeping changes small and easy to undo.”

“Virtual environments make it much easier than it was in the past to do this. We run production software in containers, and reliably build exactly the same containers for testing, even in a developer’s workspace. It’s worth the effort and cost to do this, the price is usually small compared to hunting down a single bug that crawled out of the hole created by environment mismatches.”

I agree with this part, without qualification. At least as a goal.

“Being able to automatically revert also reduces a lot of the tension of deployment, encouraging people to deploy more frequently and thus get new features out to users quickly. Blue Green Deployment allows us to both make new versions live quickly, and to roll back equally quickly if needed, by shifting traffic between deployed versions.”

What about data schemas? What about if you don’t have a product that deploys on a web server or app store? I understand that there are solutions to this, but I wonder how great a fit they are to many teams? If your team is accustomed to SQL programming—or if you already have a suite of products that use SQL databases—then how worthwhile to your business is it to prioritize moving away from SQL to a local DB like SQLite, a NoSQL document store like RavenDB, or even to a completely different back-end like Rama?

“Continuous Integration effectively eliminates delivery risk. The integrations are so small that they usually proceed without comment. An awkward integration would be one that takes more than a few minutes to resolve.”

It sounds like very much like it prioritizes eliminating delivery risk over all else. It is only applicable to products built in this way from the beginning.

“Having to put work on a new feature aside to debug a problem found in an integration test [or] feature finished two weeks ago saps productivity.”

So does constantly integrating, though! It can be noise. It’s like the noise of micro-reviewing AI responses. You have to figure out the sweet spot for your team and iterate toward that goal, always ensuring that your team can deliver even if the dream process is not already in place. Make a diagram of all the facets and discuss a plan for your project. Pragmatic. Realistic.

I don’t get the impression that Fowler is discussing a dream scenario toward which one works, but rather what he considers to be the absolute minimum process that anyone should be utterly embarrassed about themselves for not already having. I didn’t see a single sentence in this 40-page, at-times repetitive document about how to actually get there from here—or whether that’s really appropriate for many projects that people who read Martin Fowler might be working on.

“They found that elite teams deployed to production more rapidly, more frequently, and had a dramatically lower incidence of failure when they made these changes. The research also finds that teams have higher levels of performance when they have three or fewer active branches in the application’s code repository, merge branches to mainline at least once a day, and don’t have code freezes or integration phases.”

What if you don’t have an elite team?

“A two week refactoring session may greatly improve the code, but result in long merges because everyone else has been spending the last two weeks working with the old structure. This raises the costs of refactoring to prohibitive levels. Frequent integration solves this dilemma by ensuring that both those doing the refactoring and everyone else are regularly synchronizing their work.”

Some refactoring can’t just be done in mini bites like that. Sometimes, you work on a POC that takes more time to verify. Now what? Throw it away and build it from scratch in bite-sized pieces? Or integrate a long-lived branch, which is verboten?

I’m working on a sweeping change to the way solutions are configured. It involves changing packages and versions in four different solutions. Should I have merged to master everywhere and involved the whole team in my project? That sounds stupid. Sure, it takes longer to verify and integrate in one big chunk, but it has the advantage that it didn’t make upgrading the solution format the number-one priority for all developers for a sprint or two.

“[…] teams that spend a lot of effort keeping their code base healthy deliver features faster and cheaper. Time invested in writing tests and refactoring delivers impressive returns in delivery speed, and Continuous Integration is a core part of making that work in a team setting.”

For non-legacy projects. Continuous delivery can only really work for web-based products or apps. A lot of other products have to be deployed to processes that aren’t as easy to update five times a day.

“Continuous Integration is more suited for team working full-time on a product, as is usually the case with commercial software. But there is much middle ground between the classical open-source and the full-time model. We need to use our judgment about what integration policy to use that fits the commitment of the team.”

That is the first time that he’s conceded that maybe there are use cases to which this whole article doesn’t apply very well.

“If a team attempts Continuous Integration without a strong test suite, they will run into all sorts of trouble because they don’t have a mechanism for screening out bugs. If they don’t automate, integration will take too long, interfering with the flow of development.”

No kidding. You need some serious test coverage to continuously integrate and deploy. I also wonder about the size of the product you can legitimately do this. Can you imagine if your test suite takes ten minutes to run and you integrate three or four times per day? Can you imagine how much time you’re not developing software because you’re integrating someone else’s code? I understand that this happens eventually, but I wonder about the wisdom of prioritizing integration seemingly above all else.

“Continuous Integration is about integrating code to the mainline in the development team’s environment, and Continuous Delivery is the rest of the deployment pipeline heading to a production release.”

This is a good definition and I wonder that he rewrote this whole essay and didn’t put this right at the top.

“Continuous Integration ensures everyone integrates their code at least daily to the mainline in version control. Continuous Delivery then carries out any steps required to ensure that the product is releasable to product[ion] whenever anyone wishes. Continuous Deployment means the product is automatically released to production whenever it passes all the automated tests in the deployment pipeline.”

Also excellent definitions that make the distinction clear. Continuous Delivery is the one that many teams could strive for, even if they will never be able to do Continuous Delivery. The question is: at what cost?

“Those who do Continuous Integration deal with this by reframing how code review fits into their workflow.”

Well, that’s an interesting statement. Integration trumps review? Get your code in there and review later? Trust in your tests? Are you kidding me? You should review design, as well as implementation. If everyone’s coding and committing and pushing in hours, when do they review? Is the idea to have people communicate with each other only when they’ve already built something?

The article The web just gets better with Interop 2024 by Jen Simmons (Webkit Blog) writes,

“The Interop project aims to improve interoperability by encouraging browser engine teams to look deeper into specific focus areas. Now, for a third year, Apple, Bocoup, Google, Igalia, Microsoft, and Mozilla pooled our collective expertise and selected a specific subset of automated tests for 2024.

“Some of the technologies chosen have been around for a long time. Other areas are brand new. By selecting some of the highest priority features that developers have avoided for years because of their bugs, we can get them to a place where they can finally be relied on.”

When we complain about features that remain unimplemented in browsers, we also have to acknowledge that there’s only so much you can do with a given team. There are problems that are technically easier to solve than others. When we complain, we’re actually more concerned about the prioritization of issues. We want to be able to influence what gets fixed when, rather than just having to passively hope that the manufacturer eventually gets around to it.

That where the Web Platform Tests come in. The Interop 2024 project follows on iterations from 2023, 2022, and 2021, when it all started.

Last year was a banner year. For CSS “Subgrid, Container Queries, :has(), Motion Path, CSS Math Functions, inert and @property are now supported in every modern browser.” For JavaScript, we got “Improved Web APIs include Offscreen Canvas, Modules in Web Workers, Import Maps, Import Assertions, and JavaScript Modules” across all modern browsers.

These are all super-important features. E.g., Import Assertions for JSON import and Modules in Web Workers, which allows modern and modular programming, making it much easier to offload work, as one would with code running directly on modern operating systems.

What’s on the schedule for 2024?

• Although there was a lot of progress made on CSS nesting last year, it’s back on the radar this year to finalize the implementations.
• @property will similarly be more polished, as the percentage support is still quite low in many browsers.
• It’s great to see accessibility improvements for many of these features—like how sub-grids or display: contents affect element order—as this means that we will get sites that are automatically accessible, as long as we build our sites logically.
• Improvements to IndexedDB will make it easier to write powerful local-first applications (even though something like Automerge might be a better fit for apps offering concurrent or collaborative editing).
• Browser- and standards-level support for popover with anchors is long overdue, as making usable tooltips and popups is an area fraught with custom code and half-baked solutions. It’s nice to see this become an area where you’ll no longer need custom JavaScript.
• Relative Color Syntax continues the excellent trend of allowing us to write CSS without the support of a CSS preprocessor. With relative colors, dark/light theming support, CSS nesting, and CSS variables, I can’t think of a reason I would use a CSS preprocessor anymore. I know some people have used them for so much more, but I’ve not done so, so my needs are already covered, even without this extension that allows conversion between colorspaces.
• @starting-style will fill a gap in CSS that finally allows sites to indicate how an element will transition from or to display: none.

See the original article for much more detail.
 

I published a very similar version of the following article in the DevOps Wiki at Uster Technologies AG. Since nearly all of that post is general knowledge that I would have been happy to find before I started my investigations, I’m sharing it here.

Overview

When we think about navigating or debugging our code, we usually focus on the code we’ve written ourselves—local sources in our file system. IDEs have classically focused on being able to debug and navigate this code.

More and more, though, we’re also interested in navigating and debugging our versioned and compiled dependencies:

• Internal NuGet packages
• External NuGet packages
• The Base Class Library (BCL)

Most of these are available as source code. We would ideally like to be able to navigate and debug that code just as easily as we can our own.

The following sections define file types and terminology, and then explain how these concepts apply to debugging and navigation for external sources. You can also just jump to the sections on producing or consuming packages (especially as relates to authentication for private sources).

Diagram

The following diagram provides an overview of the process of obtaining external packages, along with their symbols and source files. It looks quite complicated, but accommodates the flexibility required by various stakeholders.

Symbol and source acquisition diagram

File types

There are several types of files associated with debugging and navigation:

DLL

The executable code generally only includes executable code (instructions). It can include debugging information, but this is relatively rare.

PDB

The “program database” contains the symbol information for the executable code, which allows the debugger to map instructions back to the source code.<br><br>This includes aliases (symbols), file positions, and any other necessary mappings, including references to source code files.<br><br>Although invented by Microsoft, the PDB is an industry-standard, platform-independent, and language-independent format. See, e.g., LLVM’s The PDB File Format documentation or the Wikipedia entry.

XML

The optionally generated XML documentation. Some IDEs can use this file to enhance the developer experience while browsing the source code.

*.cs

The original source code

Design Considerations

It’s reasonable to ask why this process is so complex.

Why can’t the nupkg just include the PDB and the *.cs files?

The system was designed for use cases where most sources were closed. That has changed, but the system still reflects the original design choices. The PDB files can also add about 30% to the size of the package. The original use cases preferred to avoid using 30% more space for package downloads that didn’t need the debugging information.

Why aren’t sources included in the PDB?

Again, historically, the use cases were for providing improved stack traces with symbols, but not to provide access to closed sources. Even if the sources are partially open, access may be restricted to only some users of the packages or symbols. Having the IDE request the sources separately allows an additional authorization phase.

What about open-source?

The defaults still reflect the original use cases, which actually represent fewer and fewer packages as time goes on.

These answers aren’t particularly satisfying if your use case happens to be “make a package that has symbols for excellent stack traces and sources for excellent debugging”. At least we now have IDEs that know how to work with this system and there is a lot of automation for producing packages with the desired symbol and source-code support.

Terminology

Debugging

A developer debugs source code by interrupting execution of a program—either manually or by setting breakpoints—and then stepping through the instructions, examining the contents of symbols (variables) to investigate the runtime behavior and operation of the system.

The debugger uses the PDB to allow source-level debugging, i.e. debugging in the original source code. While debugging in “lower” formats is possible, it’s not nearly as reliable as being able to step through the code in the original source code, using the original symbols.

How does the debugger obtain the PDB for a given DLL?

• First it searches in the same directory. This is by far the most common location where symbols will be found.
• Next, it searches on all known “symbol servers” in the order that they’re declared. All DLLs and PDBs have unique identifiers that make it possible to request and download the correct file.

Once the debugger has the PDB, it has everything it needs—except the source code.

Local sources

If the PDB was generated locally, then it most likely references the source files that are still in the same locations in the file system as when it was built. In that case, the debugger easily finds the source files because they’re just at the paths that are directly referenced by the PDB.

If the PDB was not generated locally or the source-code paths do not match, then there are other tricks to find the source files. Visual Studio allows you to set “Directories containing source code” for the “Debug Source Files”

Solution Debugging Options

External sources

If the sources aren’t available locally, e.g., for a NuGet package, then there is a system called SourceLink that is extremely well-supported in the .NET world that makes it possible to easily download the source files that generated a DLL and that are referenced by its PDB.

Things to be aware of:

• The package must have been built with SourceLink enabled (see producing packages).
• The sources must be available for download in a known format and structure (e.g., Azure Git Repos).
• The IDE must know how to download, cache, and use the sources for debugging or navigation.

If the package does not support SourceLink, but the sources are available, then you can download the sources locally and use the solution-level mapping above to tell the debugger where the source files are. You can also just point the debugger to the top-level folder when it asks for the file’s location, in which case the debugger makes the entry for you.

Navigation

A developer navigates by requesting the source code for a symbol. For example, if the declared type of a variable in an open source file is the class Setting, then the developer can ask the IDE to show the source of Setting by Ctrl + clicking, by pressing F12 in Visual Studio, or by pressing Ctrl + B in Rider.

As with debugging, navigating local sources is straightforward, since the sources are in the local file system. For symbols in NuGet packages, the IDE has to be clever enough to download, cache, and use the sources.

Visual Studio on its own does not support navigating to external sources via SourceLink. Instead, it always decompiles external sources, as shown in the example below.

Visual Studio decompiles external sources on navigation

If you have ReSharper installed, then the default setting is to try as hard as possible to avoid showing a decompiled version.

You can also add “Folder Substitutions” in the “Advanced Symbol options…” for navigating to “External Sources”. The option does not seem to be available in Rider.

ReSharper External Source Options

SourceLink

SourceLink is a system that provides source files for external sources like NuGet packages for debugging or navigation. In order for this to work, you must be able to provide external sources or the client is not properly configured for debugging.

See below for troubleshooting information, especially as relates to authentication for packages and source code pulled from authenticated locations.

Decompiled code

A decompiled version of the source code is a reconstruction of the original source from the instructions and information in the DLL and PDB. When sources cannot be located for a given symbol, Visual Studio, ReSharper, and Rider will produce a decompiled version as a fallback.

This is often good enough to be able to read the code reasonably well, but it leaves certain common constructs in their “lowered” format. E.g., calls to extension methods appear as static-method calls rather than as targeted on the first parameter.

This can make debugging difficult, as the instructions don’t match the mapping. Rider has support for patching the PDB on-the-fly to allow more comfortable debugging of decompiled sources. This is, however, a fallback solution for external packages over which you have no control. It’s best to configure your packages to publish with symbols and sources available to IDEs that support them, as shown in the next section.

Producing packages

The documentation to Enable debugging and diagnostics with Source Link is thorough and tells you all you need to know about all of the options.

If you’re working with Azure DevOps Services, you should include the following package reference:

<ItemGroup>
  <PackageReference Include="Microsoft.SourceLink.AzureRepos.Git" Version="8.0.0" PrivateAssets="All"/>
</ItemGroup>

With this, you’re all set. The package is published to the Azure Artifacts, with a corresponding snupkg available on the Azure symbol server and sources available via the repository URL (subject to authorization; see below for troubleshooting).

Additional Properties

You can set a few optional properties, detailed below. Most projects won’t need to set these, but they are included to spare you the research if you see them in code examples, either in your institution’s code or online. As noted, the only line you need is the package reference shown above.

EmbedAllSources

Embeds all project source files into the generated PDB

EmbedUntrackedSources

Embeds anything that’s not included in source control (kind of unclear what they’re talking about here, though); included in IncludeSymbols

PublishRepositoryUrl

Ensures that the URL of the repository supplied by the CI server or retrieved from source control manager is available in the package information. This is off by default to prevent discovery of private URLS, but it doesn’t really matter for packages published from private sources, as they are protected by Azure DevOps (or whatever) authorization.

IncludeSymbols

Indicates that the PDB should be generated and included either with the package (if DebugType is set to embedded) or in a separate symbol package (if SymbolPackageFormat is set to snupkg). This is implied when the NuGet package Microsoft.SourceLink.AzureRepos.Git is included, as shown below.

SymbolPackageFormat

Indicates which package format to use. This is set to snupkg when the NuGet package Microsoft.SourceLink.AzureRepos.Git is included, as shown below.

See the SourceLink documentation for more details. Among other details, they also note that projects that target .NET 8 no longer need to include this support explicitly because Azure Repos are supported by default, as detailed in the readme for the SourceLink project.

“If your project uses .NET SDK 8+ and is hosted by the above providers (GitHub, Azure Repos, GitLab, BitBucket) it does not need to reference any Source Link packages or set any build properties.”

Conditional packaging

You can also include the packaging conditionally in the Directory.Build.Targets, as shown below.

<ItemGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  <PackageReference Include="Microsoft.SourceLink.AzureRepos.Git" Version="8.0.0" PrivateAssets="All"/>
</ItemGroup>

See the appendix for Directory.Build.Props and Directory.Build.Targets for more information about which variables and directives are respected in which file.

Consuming packages

For debugging

If a package has SourceLink enabled and you have access to the online repository from which it was built, then to seamlessly debug into that source code, ensure the following:

• Disable Just My Code
• Check that the PDB is available

For navigation

As noted above, Visual Studio doesn’t support navigating via SourceLink. To browse external sources with JetBrains tools, ensure the following:

• Check that the PDB is available
• Set up Authentication

Troubleshooting

Symbols not loaded

Disable Just My Code

Once you’re sure that the package supports SourceLink, then you should also make sure that the Just My Code setting is disabled.

When Just My Code is enabled, the debugger skips over any code that doesn’t correspond to source code in one of the local projects.

Is it available?

• Does the package you’ve downloaded actually include symbols (a .pdb file next to the .dll file)?
• If the PDB is not included with the package, is it available on a Symbol Server?
• If it is included, is it being copied into the output folder with the DLL?

If it’s available in the package, but is not being copied to the output folder, then if you’re using .NET 7.0 SDK or higher, you can use the build property named CopyDebugSymbolFilesFromPackages.

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  <CopyDebugSymbolFilesFromPackages>true</CopyDebugSymbolFilesFromPackages>
</PropertyGroup>

Manually load the module

Verify that the symbols for the module you’re trying to debug have been loaded. If they aren’t loaded, you can try to load symbols while debugging. For more details and a screenshot, see Just My Code debugging.

Decompiling rather than downloading

If you’re trying to navigate in code, but ReSharper or Rider keeps decompiling instead of getting the sources from SourceLink, then check your External Sources settings in ReSharper or Rider. Verify that the tool is configured to check for external sources before it tries decompiling.

If the IDE is having trouble authenticating, then you will usually see a decompiled version instead. Sometimes the code is so close to the original that it’s hard to tell; scroll to the top to see if it includes the “decompiled by JetBrains…” header.

Once the IDE has decompiled a source file, it will continue to use this cached copy until you close the tab, or sometimes you have to close and re-open the project. If you’re troubleshooting your way through this setup, then you can temporarily disable decompilation as a fallback, which avoids producing the unwanted source-code variant in the first place.

Authentication fails

Visual Studio uses the authentication associated with the logged-in user that you use to enable the IDE. This can be in a weird state if you’ve recently changed your password or your authentication token is stale or in a non-refreshable state. Try logging out and back in.

JetBrains tools (Rider, ReSharper, DotPeek, etc.), on the other hand, need to be given a token.

Configure from the notification

If the tool shows a notification indicating that authentication has failed, then do the following:

• Click Configure on the notification to show a dialog
• In the resulting dialog, set:

  • User name to your Azure login, e.g., john.doe@example.com
  • Token to an Azure PAT (click for instructions on how to create one)
  • Press the Test button to verify that it works (you should see OK 200)
  • Press Ok to save the credentials

Bug in JetBrains tools

However, there is a bug whereby JetBrains tools fail to show a notification or offer a way to enter credentials. [1] That’s going to look something like this:

Download never completes

It claims that it can download the source, but it never completes. You have to cancel the dialog. If you then look at the ReSharper Output, then you’ll see something like this:

Non-OK HTTP status code

The relevant text is at the end of the third line, which indicates that the request for the source file returned a “Non-OK HTTP status code”.

PdbNavigator: Searching for 'Example.Core.AppConfig.AppConfigKeyAttribute' type sources in C:\Users\john.doe\.nuget\packages\example.core.appconfig\4.1.0\lib\netstandard2.0\Example.Core.AppConfig.pdb
PdbNavigator: File names (1) are inferred for type Example.Core.AppConfig.AppConfigKeyAttribute
PdbNavigator: Downloader: https://dev.azure.com/example/example.Core/_apis/git/repositories/Example.Core.LabInstruments/items?api-version=1.0&versionType=commit&version=8b34c2aa672facd47e835c27152f695fa796a408&path=/Example.Core/DotNetStandard/Example.Core.AppConfig/AppConfigKeyAttribute.cs -> Non-OK HTTP status code

Configure from the Credentials Manager

The most reliable way to fix this is to create the credentials in the Credential Manager. Be aware that you will need to create an Azure PAT (personal access token).

• Open Credentials Manager
• Switch to Windows Credentials
• Scroll until you see JetBrains SourceLink https://dev.azure.com/exampleOrganization

JetBrains entry in Credentials Manager

If you don’t have this entry, then that’s the problem. If you have it, but you still can’t get the sources, then edit the entry to have valid credentials.

To create or edit the record, do the following from the Credentials Manager:

• Press Add a generic credential

  

• In the resulting form, set:

  • Internet or network address to JetBrains SourceLink https://dev.azure.com/exampleOrganization
  • User name to your Azure login, e.g., john.doe@example.com
  • Password to an Azure PAT (click for instructions on how to create one)

Alternative: referencing projects, not packages

As you can see above, although publishing a package is relatively straightforward, there are quite a few stumbling blocks on the way to consuming the package for navigation and debugging. Once you have everything set up and working, it’s great, but … there is still one other drawback.

You can’t edit the code for packages.

This is not optimal. Optimally, we’d like to quickly verify that change to an upstream code would address an issue in downstream code without having to generate new packages. It would be great to just edit the upstream code as if it were part of your downstream solution until you’re sure that the change would address your downstream issue. At that point, you can copy the changes back to the upstream solution (where the dependency is produced), add tests, and produce a new version, being pretty certain that the change is effective.

The shortest possible developer-feedback loop with code in external packages is:

• Determine that you need to make a change to code in an external package
• Open the solution for that package
• Make the change
• Build the assembly
• Drop it into your output folder (along with the PDB)
• Build and run your solution with the updated code

If your package has dependencies or your change in the external package’s solution touches multiple packages, then you can do the following:

• Build packages for the solution locally
• Set up a NuGet source that points to that folder
• Update to the newer versions of the packages and restore from that source
• Build and run your solution with the updated code

If it get too complicated to do locally, then you can always commit, push, and have the CI generate new versions of your packages (hopefully with a prerelease version, e.g., 3.2.4-preview2)

The solutions outlined above have a reasonable turnaround time, but sometimes you want to pretend that the external packages are just internal projects instead. This basically entails:

• Downloading the project or projects corresponding to the packages that you want to be able to edit
• Including those projects into your solution
• Replacing the external package-references with project references

At that point, you can edit, debug, and navigate the code as if it were your own.

See the “Project Munging with Tools & PowerShell” section of How to Debug NuGet Packages with Symbols and Source Link Painlessly for a PowerShell script that can help you automate part of this.

Directory.Build.Props and Directory.Build.Targets

MSBuild supports including common configuration in project files. While earlier versions required all configuration to be included explicitly, modern versions include configuration files with special names automatically, greatly simplifying common configuration and reducing clutter in project files.

Mechanics

If the file is named Directory.Build.Props or Directory.Build.Targets, it is picked up automatically and included for all projects in that folder or any subfolder. If you use a different name, then you have to explicitly reference that file from a project or from another *.props or *.targets file. If you choose your own name, you don’t have to use the Build.Properties or Build.Targets convention, but it’s strongly recommended, to avoid confusion.

Directory.Build.Props

You can use a Directory.Build.Properties file to include settings for all projects in a folder or set of subfolders.

For example, the following package reference can and should be included in Directory.Build.Props:

<PackageReference Include="Microsoft.SourceLink.AzureRepos.Git" Version="8.0.0" PrivateAssets="All"/>

Directory.Build.Targets

If you want to include settings conditionally based on build configuration (e.g., Configuration or Platform), then you’ll have to use the Directory.Build.Targets file, which has access to those variables.

<ItemGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
  <PackageReference Include="Microsoft.SourceLink.AzureRepos.Git" Version="8.0.0" PrivateAssets="All"/>
</ItemGroup>

Rules-of-thumb

• Once you’re using the SDK-style format for project files [2], you can aggressively consolidate common settings in a Directory.Build.Props file at the root of the solution.
• If you have groups of projects to which different settings should be applied, then consider splitting those off into corresponding subfolders (e.g., “Tests”) so that you can apply those common settings with a configuration file that applies only to that folder.
• If you can’t or don’t want to move projects into subfolders, then you can create a custom `props` file and manually include it in the project files that need it.

The article Exploring Generative AI by Birgitta Böckeler (MartinFowler.com) is chock-full of helpful tips from eight newsletters totaling 25 pages that she wrote throughout 2023. I include some of my own thoughts, but most of this article consists of citations.

A lot of my analysis and notes boils down to: you need to know what you’re doing to use these tools. They can help you build things that you don’t understand, but it’s not for medium- or long-term solutions. I’ve written a lot more about the need for expertise in How important is human expertise?

“The following are the dimensions of my current mental model of tools that use LLMs (Large Language Models) to support with coding.

“Assisted tasks”

• Finding information faster, and in context
• Generating code
• “Reasoning” about code (Explaining code, or problems in the code)
• Transforming code into something else (e.g. documentation text or diagram)

“These are the types of tasks I see most commonly tackled when it comes to coding assistance, although there is a lot more if I would expand the scope to other tasks in the software delivery lifecycle.”

“In this particular case of a very common and small function like median, I would even consider using generated code for both the tests and the function. The tests were quite readable and it was easy for me to reason about their coverage, plus they would have helped me remember that I need to look at both even and uneven lengths of input. However, for other more complex functions with more custom code I would consider writing the tests myself, as a means of quality control. Especially with larger functions, I would want to think through my test cases in a structured way from scratch, instead of getting partial scenarios from a tool, and then having to fill in the missing ones.”

“The tool itself might have the answer to what’s wrong or could be improved in the generated code − is that a path to make it better in the future, or are we doomed to have circular conversation with our AI tools?”

“[…] generating tests could give me ideas for test scenarios I missed, even if I discard the code afterwards. And depending on the complexity of the function, I might consider using generated tests as well, if it’s easy to reason about the scenarios.”

“For the purposes of this memo, I’m defining “useful” as “the generated suggestions are helping me solve problems faster and at comparable quality than without the tool”. That includes not only the writing of the code, but also the review and tweaking of the generated suggestions, and dealing with rework later, should there be quality issues.”

• […]
• Boilerplate: Create boilerplate setups like an ExpressJS server, or a React component, or a database connection and query execution.
• Repetitive patterns: It helps speed up typing of things that have very common and repetitive patterns, like creating a new constructor or a data structure, or a repetition of a test setup in a test suite. I traditionally use a lot of copy and paste for these things, and Copilot can speed that up.

Interesting. I’ve just always used the existing templates or made my own expansion templates. At least then it makes exactly what I want—and even leaves the cursor in the right position afterwards.

Another thought I had is that the kind of programmer that this helps doesn’t use any generalization for common patterns. Otherwise, the suggestions wouldn’t be useful because they can’t possibly take advantage of those highly specialized patterns. Or maybe they can, if they’re included in the context. It seems unlikely, if only because the sample size is too small to be able to influence the algorithm sufficiently. But maybe enough weight can be given to the immediate context to make that work somehow.

At that point, though, you’re just spending all of your time coaxing your LLM copilot into building the code that you already knew you wanted. This practice seems like it would end up discouraging generalization and abstraction—unless it can grok your API (as I’ve noted above).

This is an age-old problem that is maybe solved, once and for all. The problem is that when you generalize a solution, it becomes much easier, more efficient, and more economical to maintain, but it can end up being more difficult to understand. If the API is well-made and addresses a problem domain with a complexity that the programmer is actually capable of understanding, then the higher-level API may be easier to use, and perhaps even maintain.

However, a non-generalized solution is sometimes easier for a novice or less-experienced programmer to understand and extend. It’s questionable whether you’d want your code being extended and maintained by someone who barely—or doesn’t—understand it, but that situation is sometimes thrust on teams and managers.

“This autocomplete-on-steroids effect can be less useful though for developers who are already very good at using IDE features, shortcuts, and things like multiple cursor mode. And beware that when coding assistants reduce the pain of repetitive code, we might be less motivated to refactor.”

“You can use a coding assistant to explore some ideas when you are getting started with more complex problems, even if you discard the suggestion afterwards.”

“The larger the suggestion, the more time you will have to spend to understand it, and the more likely it is that you will have to change it to fit your context. Larger snippets also tempt us to go in larger steps, which increases the risk of missing test coverage, or introducing things that are unnecessary.”

On the other hand,

“[…] when you do not have a plan yet because you are less experienced, or the problem is more complex, then a larger snippet might help you get started with that plan.”

This is not unlike using StackOverflow or any other resource. There’s no getting around knowing what you’re doing, at least a little bit. You can’t bootstrap without even a bootstrap.

“Experience still matters. The more experienced the developer, the more likely they are to be able to judge the quality of the suggestions, and to be able to use them effectively. As GitHub themselves put it: “It’s good at stuff you forgot.” This study even found that “in some cases, tasks took junior developers 7 to 10 percent longer with the tools than without them.””

“Using coding assistance tools effectively is a skill that is not simply learned from a training course or a blog post. It’s important to use them for a period of time, experiment in and outside of the safe waters, and build up a feeling for when this tooling is useful for you, and when to just move on and do it yourself.”

This is just like any other tool. There is no shortcut to being good at something complex. The only tasks for which there are shortcuts are the non-complex ones. In that case, you should be asking yourself why your solutions involve so much repetitive programming.

“We have found that having the right files open in the editor to enhance the prompt is quite a big factor in improving the usefulness of suggestions. However, the tools cannot distinguish good code from bad code. They will inject anything into the context that seems relevant. (According to this reverse engineering effort, GitHub Copilot will look for open files with the same programming language, and use some heuristic to find similar snippets to add to the prompt.) As a result, the coding assistant can become that developer on the team who keeps copying code from the bad examples in the codebase.”

That will be so much fun, especially if you can get an echo chamber of lower-skilled programmers approving each other’s pull requests. 😉

“We also found that after refactoring an interface, or introducing new patterns into the codebase, the assistant can get stuck in the old ways. For example, the team might want to introduce a new pattern like “start using the Factory pattern for dependency injection”, but the tool keeps suggesting the current way of dependency injection because that is still prevalent all over the codebase and in the open files. We call this a poisoned context , and we don’t really have a good way to mitigate this yet.”

“Using a coding assistant means having to do small code reviews over and over again. Usually when we code, our flow is much more about actively writing code, and implementing the solution plan in our head. This is now sprinkled with reading and reviewing code, which is cognitively different, and also something most of us enjoy less than actively producing code. This can lead to review fatigue, and a feeling that the flow is more disrupted than enhanced by the assistant.”

“Automation Bias is our tendency “to favor suggestions from automated systems and to ignore contradictory information made without automation, even if it is correct.” Once we have had good experience and success with GenAI assistants, we might start trusting them too much.”

“[…] once we have that multi-line code suggestion from the tool, it can feel more rational to spend 20 minutes on making that suggestion work than to spend 5 minutes on writing the code ourselves once we see the suggestion is not quite right.”

“Once we have seen a code suggestion, it’s hard to unsee it, and we have a harder time thinking about other solutions. That is because of the Anchoring Effect, which happens when “an individual’s decisions are influenced by a particular reference point or ‘anchor’”. so while coding assistants’ suggestions can be great for brainstorming when we don’t know how to solve something yet, awareness of the Anchoring Effect is important when the brainstorm is not fruitful, and we need to reset our brain for a fresh start.”

“The framing of coding assistants as pair programmers is a disservice to the practice, and reinforces the widespread simplified understanding and misconception of what the benefits of pairing are.”

“Pair programming however is also about the type of knowledge sharing that creates collective code ownership, and a shared knowledge of the history of the codebase. It’s about sharing the tacit knowledge that is not written down anywhere, and therefore also not available to a Large Language Model. Pairing is also about improving team flow, avoiding waste, and making Continuous Integration easier. It helps us practice collaboration skills like communication, empathy, and giving and receiving feedback. And it provides precious opportunities to bond with one another in remote-first teams.”

“LLMs rarely provide the exact functionality we need after a single prompt. So iterative development is not going away yet. Also, LLMs appear to “elicit reasoning” (see linked study) when they solve problems incrementally via chain-of-thought prompting. LLM-based AI coding assistants perform best when they divide-and-conquer problems, and TDD is how we do that for software development.”

“Some examples of starting context that have worked for us:”

• ASCII art mockup
• Acceptance Criteria
• Guiding Assumptions such as:

  • “No GUI needed”
  • “Use Object Oriented Programming” (vs. Functional Programming)

“For example, if we are working on backend code, and Copilot is code-completing our test example name to be, “given the user… clicks the buy button ” , this tells us that we should update the top-of-file context to specify, “assume no GUI” or, “this test suite interfaces with the API endpoints of a Python Flask app”.”

“Copilot often fails to take “baby steps”. For example, when adding a new method, the “baby step” means returning a hard-coded value that passes the test. To date, we haven’t been able to coax Copilot to take this approach.”

Knowing a bit about how LLMs work, there’s no way you really could train it to do TDD, because it’s an iterative process. It doesn’t know what TDD is, nor does the way it’s built have any mechanism for learning how to do it. Nor does it know what coding is, for that matter. It’s just a really, really good guesser. Everything it does is hallucination. It’s just that some of it is useful.

“As a workaround, we “backfill” the missing tests. While this diverges from the standard TDD flow, we have yet to see any serious issues with our workaround.”

Changing how you program because of the tool is something you should do deliberately. This is a slippery slope.

“For implementation code that needs updating, the most effective way to involve Copilot is to delete the implementation and have it regenerate the code from scratch. If this fails, deleting the method contents and writing out the step-by-step approach using code comments may help. Failing that, the best way forward may be to simply turn off Copilot momentarily and code out the solution manually.”

Jaysus. That’s pretty grim.

“The common saying, “garbage in, garbage out” applies to both Data Engineering as well as Generative AI and LLMs. Stated differently: higher quality inputs allow for the capability of LLMs to be better leveraged. In our case, TDD maintains a high level of code quality. This high quality input leads to better Copilot performance than is otherwise possible.”

“Model-Driven Development (MDD). We would come up with a modeling language to represent our domain or application, and then describe our requirements with that language, either graphically or textually (customized UML, or DSLs). Then we would build code generators to translate those models into code, and leave designated areas in the code that would be implemented and customized by developers.”

“That unreliability creates two main risks: It can affect the quality of my code negatively, and it can waste my time. Given these risks, quickly and effectively assessing my confidence in the coding assistant’s input is crucial.”

“Can my IDE help me with the feedback loop? Do I have syntax highlighting, compiler or transpiler integration, linting plugins? Do I have a test, or a quick way to run the suggested code manually?”

“I have noticed that in CSS, GitHub Copilot suggests flexbox layout to me a lot. Choosing a layouting approach is a big decision though, so I would want to consult with a frontend expert and other members of my team before I use this.”

That’s because you care about architecture. Review was always important, but more so when code is being written by something you never hired.

“How long-lived will this code be? If I’m working on a prototype, or a throwaway piece of code, I’m more likely to use the AI input without much questioning than if I’m working on a production system.”

“[…] it’s also good to know if the AI tool at hand has access to more information than just the training data. If I’m using a chat, I want to be aware if it has the ability to take online searches into account, or if it is limited to the training data.”

“To mitigate the risk of wasting my time, one approach I take is to give it a kind of ultimatum. If the suggestion doesn’t bring me value with little additional effort, I move on. If an input is not helping me quick enough, I always assume the worst about the assistant, rather than giving it the benefit of the doubt and spending 20 more minutes on making it work.”

“GitHub Copilot is not a traditional code generator that gives you 100% what you need. But in 40-60% of situations, it can get you 40-80% of the way there, which is still useful. When you adjust these expectations, and give yourself some time to understand the behaviours and quirks of the eager donkey, you’ll get more out of AI coding assistants.”

The latest video by Nick Chapsas has a more-than-usually clickbait-y headline. The “big” problem that NativeAOT has, is that it’s 4% slower during runtime than the JIT-compiled version.

NativeAOT in .NET 8 Has One Big Problem by Nick Chapsas (YouTube)

That doesn’t seem like such a big problem to me, when the point of AOT is to improve cold-start times for applications launched on-demand. For that use-case, AOT shines. It’s over 4x faster on startup than the JIT-compiled version. It’s incredibly impressive that JIT-compilation takes less than 1/10 of a second, but it’s still 4x slower than AOT.

Spider graph of AOT vs. JIT

So, you get the app started 4x fast, but it then performs 4% more slowly than the non-AOT version. It really depends on the use-case, but it’s great for the common one of starting a server to answer a function call—think Azure Functions or AWS Lambdas—and then shutting down again, possibly immediately.

Damian P Edwards (Principal Architect at Microsoft) commented on the post,

“[There are a] few things that cause the slightly lower performance in native AOT apps right now. First (in apps using the web SDK) is the new DATAS Server GC mode. This new GC mode uses far less memory than traditional ServerGC by dynamically adapting memory use based on the app’s demands, but in this 1st generation it impacts the performance slightly. The goal is to remove the performance impact and enable DATAS for all Server GC apps in the future.

“Second is CoreCLR in .NET 8 has Dynamic PGO enabled by default, which allows the JIT to recompile hot methods with more aggressive optimizations based on what it observes while the app is running. Native AOT has static PGO with a default profile applied and by definition can never have Dynamic PGO.

“Thirdly, JIT can detect hardware capabilities (e.g. CPU intrinsics) at runtime and target those in the code it generates. Native AOT however defaults to a highly compatible target instruction set which won’t have those optimizations but you can specify them at compile time based on the hardware you know you’re going to run on.

“Running the tests in [the] video with DATAS disabled and native AOT configured for the target CPU could improve the results slightly.”

To summarize:

1. The DATAS GC mode is in-use for AOT, but still being fine-tuned.

2. An AOT-compiled app cannot benefit from dynamic PGO. It benefits from static PGO, but cannot recompile itself on-the-fly because it doesn’t have a JIT compiler to do so.

   The JIT-compiled app can dynamically recompile what it observes as performance hotspots with more highly optimized code. I wrote a bit about how Safari does something similar for JavaScript in Optimizing compilation and execution for dynamic languages—although for JavaScript, dynamic recompilation is sometimes necessary for backing out of an incorrect assumption about what type a variable is going to have.

3. As well, a JIT-compiled app can take actual hardware capabilities into account, while an AOT-compiled app necessarily targets a static hardware profile.

   The generic hardware profile is going to be extremely conservative about capabilities because if it assumes a capability that doesn’t exist, the app simply won’t run. Choosing a hardware profile for AOT that matches the target hardware would boost performance.

I guess that was more of a rephrasing, rather than a summary.

Anyway, another commenter asked,

“[…] would it be possible in the future for a JIT application with Dynamic PGO that has run for a while and has made all kinds of optimizations to then create a “profile” of sorts that could be used by the Native AOT compiler to build an application that is both fast in startup time and highly optimized for a given workload?”

Yes. That should be possible. It’s unclear what sort of extra performance boost this would give, especially if you’d already fine-tuned the target hardware profile—which is the first thing you should do. I could imagine adding this sort of profiling as a compilation step, though. You always have to be careful, though, whenever you’re running something in production that is different than what you’ve tested. We put a lot of faith in the JIT and dynamic PGO, don’t we?

I wanted to also note that, at the end of the video, Chapsas showed Microsoft’s numbers, which confirm the performance drop, but also show an over 50% reduction in working set! Dude! How do you not mention that!? The app uses less than half of the memory and runs almost as fast? Yes, please! That’s a huge win for people paying for cloud-based services.

For once, I’m somewhat surprised to see how naive Nick’s take is—that a 4% drop in performance is at-all significant, especially when the “slow” version is still processing 50,000 requests per second in a performance-constrained environment. He did mention a trade-off, but was very excited to tell people that AOT is slower during runtime.

There are always trade-offs and you should be very aware of the actual non-functional requirements for your application before you decide whether to use a technology or not. For 99.9% of the applications, the 4% drop in performance vis á vis a JIT-compiled version won’t be the deciding factor. When it’s accompanied by a working set that’s only ½ the size, then it becomes an even more attractive target.
 

A build started started failing after a commit. However, the errors had nothing to do with the changes in the commit. A little investigation revealed that the cloud agent had started using a newer version of the build tool that included an expanded set of default warnings. These warnings started appearing first on CI because developers hadn’t had the chance to update their tools yet.

The “warnings as errors” setting turned what would have been a build with a few extra warnings into a failing build that prevented a developer from being able to apply completely unrelated changes. The setting allowed new, unrelated, and irrelevant warnings to push their way to the top of the priority queue.

👉 tl;dr: I don’t think we should use the “errors as warnings” setting anymore. You can get the same benefit —and even more—by using newer, more finer-grained configuration options.

What are we trying to accomplish?

This section wasn’t included in my original draft of this essay. It only occurred to me under the shower that this is the real reason why I wrote a ten-page essay to answer a teammate’s question in a PR review.

In hindsight, it’s obvious: to answer whether we should re-enable the “warnings as errors” setting, we should first think about what doing so would accomplish. What need does it fulfill?

The rest of this essay meanders drunkenly along a path toward what I hope is a reasonable answer.

My team doesn’t care about warnings

I understand the sentiment. You’re in a team that never, or rarely, looks at warnings. You’ve given up on teaching them how to look at warnings and keep them fixed. Fine. You just make every warning an error and now they absolutely have to fix everything. Problem solved.

My team now only cares about warnings

Except it isn’t, is it? Not really.

It’s like forcing people to wash their dishes while they’re still eating their meal.

What you’ve now done is ensured that your team will be constantly fixing errors that aren’t really errors at times when they wouldn’t want or need to be doing so. It’s like forcing people to wash their dishes while they’re still eating their meal. [1]

Don’t make me waste time pretty-printing code that I’m still writing! How annoying is it when you can’t run a test because your comment has an extra line below it? Are you kidding me? [2]

My team is not homogenous [3]

When the runtime changes or any external dependencies change, you’ll have to fix all of the new warnings because you’re making them into errors. Instead of leaving yourself time to properly address the warnings, it immediately becomes your highest priority because you can no longer work.

The practice leaves your team in a position where it might lose the runtime lottery, When the runtime changes, you won’t have control which person will be forced have to make those fixes so they can continue working. It may be the wrong person in your team, someone who’s going to choose the quickest way to get compiling again rather than the best way to incorporate the new errors and warnings and inspections.

How much do you trust your team?

If your team does care about warnings, then, … why do you need to make them errors?

Before handcuffing developers with a setting, think about whether there isn’t a trust problem first. Are you addressing a symptom rather than the cause?

While it’s possible that applying handcuffs is the best possible solution in your case, consider that there are other solutions along a spectrum that goes from “enforcing discipline” to “relying on individual discipline”.

Any feature that’s enforced at all times will end up hampering efficiency and flexibility in some cases, while any feature that’s left up to developers is liable to not be applied consistently.

The job of the person setting up code-style configuration is to thread that needle, tailoring the configuration for the team and solution at hand.

If you have a lot of solutions and teams, then you also get to consider the maintenance overhead of having too many custom configurations. In that case, you might want to make a few standard bundles that group teams and solutions, like “legacy”, “modern”, “junior team”, etc.

You don’t have to name them like that, but the name should give you an idea of how loose or restrictive the settings would be.

Let the CI do it, then

I don’t have time for all of that. Let’s just run them on the CI. Warnings as errors in the cloud FTW!

Now you’re allowing team members to push all the way up to the server before they realize that they have errors. Granted, they’re actually warnings, but you can’t merge to master until you fix them, so, yeah, they’re errors. This isn’t less annoying.

But, but, but, what if they’re, like, real warnings? Like “possible NullReferenceException” or something like that? That’s a good point, sure.

But, in most cases, it’s something more like “extra line found at end of file”, “space missing after parenthesis”, “method can be made private”, “class should be internal”, etc.

There are better—more automated—ways of addressing some of those, which we’ll discuss below.

The CI is not necessarily stable

Also, what if some warnings start appearing in your CI because of a tooling change? That can never happen, though, right? Because you’ve locked down all of your tool versions so that it can never happen? No? You didn’t do that? You’re using “latest”? Why?

The people building the tools are pretty clever, so we want to know what new things they have to tell us about our code.

Oh, right. Because it makes sense. If you lock down your tool versions, you run the very real risk of not knowing when your build will stop running with more-modern tools. You run the risk of it having been years since you last changed anything in your build and your being stuck with those settings and old tools … until they’re obsolete or no longer available on your build server.

It’s better to use “latest” and have an occasional spike of warnings than to just never know where you stand with newer toolchains. Locking down tool versions leads to things like DevOps having to set up on-site build agents with Visual Studio 2010 on them for certain projects.

OK, so we want to use latest tools, but that means that we might also get new warnings. These are a good thing! The people building the tools are pretty clever, so we want to know what new things they have to tell us about our code.

The future broke the build

What we don’t want is for those new things to break builds that used to be running just fine.

This usually shows up when someone pushes new commits, runs the CI, and sees that they’re getting errors that they didn’t see locally. WTH? “My code didn’t cause those errors?”

The drawback here is this is (A) annoying and (B) it’s very possible that the new errors are a distraction at this point in time. The person’s bug fix may be important, but the new warnings have now bumped themselves to the top of the priority queue!

And what if the person whose build has failed isn’t well-qualified to address these new warnings? Well, then they get to bump the new warnings to the top of someone else’s priority queue! Probably a more senior developer. Fun for all!

What’s the solution then? Well, if you realize that the new warnings appeared because of a tool change, then I suppose you should try to pin the tool version on the CI, with all of the drawbacks outlined above.

That’s assuming that the person to whom this happens is (A) capable of figuring this out and (B) knows how to pin the tool version. And (C) we don’t really like that solution, for the reasons outlined above.

What were the requirements again?

What about if we think again about what we’re trying to accomplish with “warnings as errors”?

Thinking…🤔🤔🤔…

The system must allow individual configuration of severity.

We want certain warnings to be errors.

The system must not require all team members be capable of configuring it.

We want clever tool people to configure things for maximum developer comfort, warning visibility with everyone having to become a clever tool person (which isn’t generally possible).

The system must be configurable per solution.

Each solution should be able to decide what is an error and what is a warning and what is a suggestion. You can’t make “possible null-reference exception” an error in some legacy solutions without completely killing forward progress.

We want warnings to indicate potential problems, but be careful about forcing a solution to address all of them immediately. It’s more realistic to create tasks to slowly eliminate warnings, only switching a setting to an error later, to prevent future transgressions.

The system must have a versioned configuration.

We want the configuration to grow with the project, so older versions have their own configuration, with which they can still be run when needed.

The system should use default settings wherever possible

No matter how clever your own team’s tool person is, the people who designed the whole thing are probably even better. Apply configuration changes knowingly and judiciously.

The system should not force a developer to change priorities unnecessarily.

If the developer is focused on something, they shouldn’t be forced to switch modes and prioritize formatting. Use gentle, visible hints, unless it’s really, really relevant to what they’re working on.

For example, a possible NullReferenceException is something to be avoided, but is it really an error in all code? It’s definitely a warning, but if the developer knows that it doesn’t matter right now, then they should be able to ignore it, no?

I mean, they haven’t even committed it yet (as far as you know 😉). Maybe they have a breakpoint to see how the heck that variable could be null in the first place and they were just going to bounce the EIP past the crash anyway. YOLO.

Anyway, we want to be really careful about how pushy we are with the IDE configuration. We want to strike a balance between missing actual problems and decreasing efficiency. We don’t want the developer above to have to write a suppression—or, even worse, do some other, ad-hoc short-circuit of inspections—in order to keep working.

The system should be future-proof

We don’t want running builds to stop running just because we’ve upgraded tools, but made no changes to the tools. This won’t always be possible but, in this case, the “warnings as errors” setting is a pretty obvious footgun.

The system should provide developers everything they need locally to avoid CI failure

Something should fail only on CI as a last resort. That is, a developer must have tools that make it relatively easy to pass CI. This includes being able to see all warnings in the solution, whether warnings would fail the CI, or having an easy way to apply formatting to all files, if incorrect formatting would fail the build.

We want to avoid a process that leads to half of our commits being called “fix formatting” and “remove warnings”. So, we should consider things like having the IDE auto-reformat files on save.

The system should discourage allowing inspection violations to be committed

Inspections should be applied and made visible as quickly as possible, to give the developer the opportunity to produce conforming code from the get-go. The path of least resistance should result in committing code that will also pass CI.

We don’t want to encourage “noisy” commits that “fix up” formatting or other inspection violations. We would rather have a high signal-to-noise ratio in our commits. We want compact, descriptive commits—so we don’t want bug-fix commits to include formatting changes to other parts of the file, if we can avoid it.

Looking at these requirements, we have to conclude that the “warnings as errors” configuration option is an absolute cudgel that we had to use in the old days because we didn’t have fine-tuned control of the inspection-configuration.

We are no longer in the dark ages

Can we do better today, with modern tools?

Absolutely, we can! Most modern IDEs support .editorconfig, which allows fine-tuned configuration of both code-style and formatting, especially for languages like C# and TypeScript/JavaScript. The wide variety of JetBrains, Intellij-based tools use it as well, e.g. PyCharm, WebStorm, or PHPStorm. Visual Studio understands it. Visual Studio Code understands it.

Of course, the devil is in the details and, the degree to which code-inspection configuration applies from one IDE to another depends very much on the level of standardization for that language and environment. The .NET/C# world has a high degree of standardization, which is very helpful.

Using EditorConfig

EditorConfig allows you to control almost anything you can think of about your code style or formatting. These are called inspections, each of which you can configure with an inspection-specific value and a severity to assign when the inspection is triggered.

For example:

dotnet_style_require_accessibility_modifiers = for_non_interface_members:silent
dotnet_style_prefer_auto_properties = true:silent

The two inspections above should be relatively obvious. In both cases, the preferred setting is configured, but the severity is “silent”, so the IDE doesn’t complain about it.

What’s the point of configuring a preference and then not showing it to the developer?

Ah, because the developer is the not the only one modifying the code.

Excuse me?

The IDE also writes code

Don’t forget that the IDE will auto-format the code when requested. The IDE also writes code when it refactors anything. It needs to know how to format the code that it’s inserting or modifying.

The IDE uses the configuration in the EditorConfig to determine how to format the code. Your tools guy can configure the EditorConfig to conform to the style that the solution / team wants to use. When the code is auto-formatted or refactored, everything should end up looking just the way they wanted it.

How to apply “silent” inspections?

If you have a “silent” severity, that means it’s something that you don’t want the team wasting time with during development. However, if no-one ever auto-formats the code, then those inspections will never be applied.

You should consider the process by which your solution will be made to conform with silent inspections in the EditorConfig.

Visible inspections

If the inspection severity is suggestion or higher [4], then the developer sees an indicator in the code when the file is open.

Suggestions, warnings, and errors are shown in the build output, as well. Of course, the developer can disable showing warnings and messages (where suggestions appear) in the error-list pane, but you can’t control everything—and you shouldn’t try.

Give your developers the tools and configuration to be efficient and produce good code, but try not to be too pushy about when they do it.

Invisible inspections

If the inspection severity is silent or none, then the inspection setting is only used by auto-formatting and refactoring tools.

In this case, you’ll have to consider when will your code be formatted? Do your developers occasionally auto-format files? Do they auto-format on save? Is there a step in the CI that auto-formats everything before compilation? If so, does it commit those changes? Or does the CI reject for formatting warnings?

If you have silent inspections, be honest about when they’re going to be applied. If you don’t have a plan, then they will be applied seemingly randomly when someone inadvertently triggers the hotkey for auto-formatting a file [5], which may lead to unpleasant surprises and/or messy commits.

Code Style vs. Formatting

Let’s clear up the distinction between these two main groups of inspections.

Code style

A code-style inspection expresses a preference for something that affects semantic content. That is, applying the fix for the inspection may change the code in a way that makes it compile differently or may lead to it not compiling at all. Even something as subtle as using var instead of an explicit type can, in very rare cases, lead to code that no longer compiles. By now, many IDE tools are generally clever enough to avoid even suggesting such a change, but it can still happen.

Formatting

A formatting inspection expresses a preference that affects only syntactic content. That is, applying the fix for the inspection will change the appearance, but will not change how the code compiles. Consistent formatting is very important for direct readability of the code, but also to avoid spurious differences in files when inspecting commits. The fewer differences there are, the less likely it is that conflicts appear when merging or rebasing.

Local vs. CI

So we’ve examined inspections in detail and talked a lot about setting severity to optimize the developer feedback loop i.e., we don’t want to mess with a developer’s priority queue unless absolutely necessary.

But aren’t there some things that we might allow a developer to do locally but not allow to pass CI? That’s where the “warnings as errors” setting ensured that the CI never passed, even if the developer forgot to check something locally. For example, it’s important to have consistent formatting before attempting a merge.

There are other ways to encourage and support proper coding practices, though.

Auto-formatting files

Most IDEs have an option to run formatting or code cleanup automatically when files are saved. Consider setting this option in your solution. Developers will become accustomed to having the IDE lightly reformat even their WIP code and they will always have correctly formatted files.

Local pre-commit hooks

Pre-commit hooks can run locally, running global formatting on the code base before a developer can commit. This is kind of touchy, as sometimes developers are just committing a WIP to avoid losing their changes. It would be annoying if you had to clean up your formatting just to commit those.

You could include auto-formatting in the commit hook, but it’s probably better to set up auto-formatting in the IDE.

Server pre-commit hooks

Instead of a local pre-commit hook, you can configure a pre-commit hook on the server. This hook could cause a push to be rejected if its head commit doesn’t conform to certain conditions.

But…isn’t that what the CI is for? Well, kind of, but the CI runs only after the commits have landed on the server. It’s prefereable to have the developer fix commits locally before being able to push, again, to avoid “fix formatting” and “cleanup warnings” commits.

You could choose which branch patterns to run these on.

My recommendation is to lean as heavily as possible on IDE configuration before getting lost in the weeds with commit hooks.

Avoiding ugly or “noisy” commits

As soon as we start talking about “fixes” for warnings or formatting, we’re talking about “noisy” commits. If we enforce inspections more strictly on CI than we do locally, then there will be more “fixup” commits.

OK, so what do we do about them?

Squash ‘em!

Right? Right?

🫠

Kind of. Look, the PR machinery allows you to merge, rebase, squash-merge, or squash-rebase. That’s OK, but it’s not great. A lot of times, you’ll have four commits that are descriptive and semantically relevant, describing changes that were made, as well as a few commits that address problems that either came up in CI or as part of the review. Don’t you think you should squash those into the four commits and make a clean history instead of just squashing the whole lump into one big hairball?

Or do you think that each PR should have only one commit, equating a branch with a commit (as e.g. plugins like Graphite positively encourage)? I recently wrote PRs suck. Stop trying to fix them. that also touches on the workflow outlined below.

You see how tool configuration affects everything? You have to think about how your team builds PRs, how they review PRs, how they repair PRs after review—or whether they even use PRs.

I would encourage a more real-time review culture, where possible.

1. Set up the tools so a developer has a good chance of committing conforming code
2. When a developer has a set of commits they want to push, they ask another team member to review, live, explaining what they did.
3. The live reviewer can point out any issues in the code and they can repair them together, all without wasting time writing and reading review comments.
4. Those repairs can be squashed into the appropriate commits before anything’s pushed to the server.
5. Once on the server, the CI runs. If something still fails, the original developer can squash in fixes and force-push [6] to keep the commits in the PR clean.
6. You can create a PR to note the integration to master, but then its mostly a formality.

Configuring your solution

Phew! So, what have we learned?

• There is no one-size-fits-all solution.
• Errors are priority interrupts. The developer cannot execute until an error is fixed. Be judicious about which inspections you promote to errors.
• “warnings as errors” is an inelegant configuration option that promotes any warning to an error. That’s not very judicious at all. It has been replaced by a much more flexible and granular system in EditorConfig.
• Encourage a culture in your team that pays attention to warnings. There will always be team members who do more than others. That’s just life. Don’t force those team members least equipped to deal with them to address warnings by making them errors. Baby steps.
• Silent inspections will only be applied when the tools apply them, e.g., with auto-formatting or refactoring. Consider when those inspections will be applied.
• Keep what’s executed locally as close as possible to what’s executed in CI
• Consider how you want your whole process to work, right up through integration to the trunk/main/master.

If that all sounds like a lot, well—it is. Building clean, maintainable code is a complex undertaking. There are a lot of tools that can help, but you have to put some time into thinking how you want to use them, and then into configuring them so they help you instead of getting in your way.

It’s a delicate balancing act: to give developers the best chance of (A) producing conforming code in the first place and (B) avoiding “noisy” commits, while (C) not hitting them with priority interrupts irrelevant to what they’re working on. There will be tradeoffs.

Sharing or copying configuration?

Once you’ve set up a couple of solutions, you can just copy/paste the configuration to others as a starting point. Remember, though, that solutions are usually pretty unique. Only consider generalizing or packaging a configuration if you’ve considered that,

• It will be more difficult for solutions that use the package to override settings that turn out not be as standard as you thought.
• It will be more time-consuming to make changes to the configuration because you have to roll out a new version—e.g., when the underlying tools change—with potentially multiple stakeholder solutions.
• This extra work may discourage solutions from improving the shared configuration. Instead, it will languish, with all users annoyed by the same inconsistencies, but no-one willing to do the work to address them.

For these reasons, each solution having its own copy of the configuration is probably better. They can just copy/paste—the horror!—improvements where appropriate. If you’re worrying about configurations drifting out-of-sync, schedule a work item every few sprints that evaluates and possibly re-syncs configurations.

Conclusion

There are always trade-offs. Improving code-quality is an incremental process. So is configuring the tools that support that process. It gets easier with practice. Good luck!

I read through the article Your GitHub pull request workflow is slowing everyone down (Graphite.Dev) with great interest because I, too, am not thrilled about how PRs work. While I agree with the problems Graphite see with PRs, I think they miss other problems—and I don’t like their solution very much.

PRs are, apparently, HUGE

“The single most important bottleneck is PR size − large PRs can make code reviews frustrating and ineffective. The average PR on GitHub has 900+ lines of code changes. For speed and quality, PRs should be maintained under 200 lines—with 50 lines being ideal. To put this in perspective, where giant 500+ line PRs take around 9 days to get merged on average, tiny PRs under 100 lines can make it from creation to landing within hours.”

Holy shit! The average is 900 lines? That’s already using the system completely incorrectly. That’s so wild. It absolutely confirms my theory that PRs are a terrible way of committing code. I already thought they were terrible just because of the limited UI and lack of introspection of what the code you’re reviewing actually does.

PRs don’t encourage starting and running the change to verify that it actually works as advertised. You’re not using any of the tools that you use to develop code to review it. How silly is that? If you load changes into an IDE, you can see how many warnings there are, see if the layout shifts when you format the document, etc. Why would you want to review in a completely different environment? As Robin Williams once eloquently put it, It’s like masturbating with an oven mitt. (YouTube).

Not only that, but people probably aren’t looking at individual commits, so they’re just reviewing 900+ lines at once. The fewer people there are looking at individual commits, the fewer people there will be who make good, individual commits. This is a shame because it would counteract the awfulness of reviewing code in the PR web-UI, at least a little bit.

PR web UIs are not good for reviews

There are far better and more efficient ways of reviewing code than with PR web UIs. Reviewing through a PR web UI should be a fallback that you only use when nothing else is possible.

If you’re in the same time zone and working on the same schedule as the rest of your team, there is absolutely no reason why you should you be using the PR web UI instead of real-time reviews of local commits.

What the current PR machinery does is fool remote, async teams into thinking that they’re reviewing code efficiently. A face-to-face, real-time review will be much more efficient and yield much higher-quality code.

I honestly can’t believe the high pain threshold that some developers have.

If the developer hasn’t pushed yet, then:

1. Review with the person who wrote it.

If the developer has pushed and is not available for real-time review, then:

1. Pull.
2. Open the branch in SmartGit.
3. Examine the commits.
4. Launch the solution/project.
5. Run the tests locally.

6. Apply your own commits instead of review notes wherever possible.

   Yes, you can do this! Why not? You’re both on the same team. It’s a shared code base, not someone’s personal zen garden. Instead of explaining what you would want changed, just make your suggestion in the form of a commit. It’s often more efficient than writing prose.

7. Add review notes in SmartGit (synced with GitHub, Azure, GitLab, etc.) or use the PR Web UI to add review notes.

You can thank me later.

Errors slip in

“Problems can easily get hidden between the diffs, and reviewers often make assumptions instead of testing to avoid feeling overwhelmed. One particularly interesting finding is that as the size of a PR increases (by number of files changed), the amount of time reviewers spend on each file decreases significantly (for PRs with 8 or more files changed).”

Obviously! But it’s good to measure—this was my intuition. PRs don’t encourage local testing or verification in an environment similar to that which the original developer used.

Dumbing down Git

“By default, every PR is restricted to only 1 commit of <200 lines, keeping changes tightly scoped. This forces developers to consciously limit work to related changes—the registration endpoint PR can’t sneak in unrelated styling tweaks.”

Yikes! I don’t like the sound of that. So you make multiple PRs rather than one PR with multiple smaller commits? Why don’t you just review commits rather than one giant blob? Do you really need to corral each commit into its own branch and PR to force yourselves to actually make useful commits?

Yeeess? 🧐

“Stacking centers around breaking down big feature work into chains of smaller pull requests. Each PR is typically limited to 1 commit focused on an isolated change. This restriction guides developers to consciously make only a single change, squashing and rebasing along the way, instead of cluttering the PR with random unnecessary commits like “typo fixes”.”

This is yet another technique invented to accommodate teams that don’t trust each other, or that contain people who, if they can’t be trained to do better—or don’t understand what better is—probably shouldn’t be programming yet. Instead of teaching team members how to use their tools, they impose an arbitrary rule. What a kindergarten.

Integrate all the time!?

“Unlike Git workflows, where it is easy to neglect staying updated, Graphite centers your workflow around continually integrating with the current mainline state.”

Yikes! I don’t love the sound of that, either. Doesn’t that force you to spend more time on integration that you might have spent working? I understand you don’t want to have long-lived branches, but now you’re just shooting to the other extreme, forcing integration on every pull.

It’s not bad as long as the integrations are automatic, but might not be appropriate for developers who aren’t great at resolving merge conflicts. Even if they know how to deal with them well, might they not waste time resolving conflicts integrating a version of their code that wasn’t at all ready to be integrated?

I understand that this feature follows from the logic of “if you integrate more often, then integration is easier,” but, again, you’re taking agency out of developers’ hands, implicitly not trusting your team members. I don’t like it.

If you have several stacked commits, I wonder how much shuffling there is in the working tree (causing unwanted IDE reloads) during the integration cascade. Are they somehow integrating without touching the working tree? I don’t know that that’s possible.

Go ahead and work on the main branch if you want—I do it all the time—but this should be more of a choice than it sounds like it is.

I remember this…

“This command will add your changes and create a new branch in one motion. You can then continue iterating by creating and stacking additional branches:”

Ah, I see now. They’ve reinvented Mercurial’s patch queues. Everything old is new again.

A really bright and good friend of mine added an extension to Mercurial’s mq decades ago that sounds like it works the same. I remember discussing the technique with him as he was developing it.

Conclusion

I’m a bit worried about two things:

1. The one-commit-per-branch thing
2. The auto-integration-cascade

“By cleaning up your PR commit history, you ensure a clear and concise main branch history that makes it easy to see exactly what’s changed over time.”

By enforcing one commit per branch, you dumb everything down.

It does seem that, instead of acknowledging that PR supremacy is stupid, Graphite doubles down, strips branches of most of their functionality by equating them to commits, and uses multiple PRs to force people to review by commit. It seems like a waste.

But, hey, maybe I need to actually try it. I might be missing something.

Still, instead of adding another tool, I think you should use git better.

• Set up your local tools—or in-cloud IDEs, whatever—to support building the kind of code you want. See How to replace “warnings as errors” in your process for more information.
• Encourage your team members to learn how to use those tools.
• Have a review culture centered on real-time reviews where quick fixes and changes can be made before you’ve even pushed anything. This cultivates a culture of respect for commits.
• Use PRs only to “stamp” a set of changes and merge them to the trunk.

The article Some notes on Local-First Development by Kyle Matthews (Bricolage) focuses on a very good trend in app development, but focuses a bit too much on what he calls DX, or developer experience.

“I see “local-first” as shifting reads and writes to an embedded database in each client via“sync engines” that facilitate data exchange between clients and servers. […] The benefits are multiple:”

• Simplified state management for developers.
• Built-in support for real-time sync, offline usage, and multiplayer collaborative features.
• Faster (60 FPS)
• CRUD
• More robust applications for end-users.

I don’t want to read too much into it, but he did mention end-users only in the last bullet point.

I think the author is focusing too much on the tech and too little on the value. DX is great and all, but it’s about the UX, no? Every app would benefit from realtime updates if it’s cheap and easy to build. Almost every app is multiplayer, if you think about it a bit.

“For almost any real-time use case, I’d choose replicated data structures over raw web sockets as they give you a much simpler DX and robust guarantees that clients will get updates.”

No, my friend. You’ve come to the right conclusion for the wrong reason.

If the tech is solid, if it doesn’t negatively influence debuggability or traceability, if it’s predictable, if operations can be correlated, if you don’t end up limiting your functionality to fit the framework—then go for it.

What I mean is that it’s important that the thought process that leads to the correct conclusion serves all stakeholders. If you’re only doing things because they’re better for developers, then, eventually, you’re going to be deciding against the users.

Be aware of the trade-offs, and be sure all of the stakeholders can live with them. What does good DX translate to for other stakeholders? Easier maintenance? Less complexity? Easier onboarding? The DX is really mostly secondary unless you’re making a framework, in which case it might matter. No-one cares about DX for real-world products. I love good DX, but I’m a developer! As a developer with a lot of experience, I’m forced to admit that it’s not at all a primary goal. Having good DX might lead to other desirable things, but that doesn’t make it directly desirable. Don’t forget that.

This is a brilliant interview, in that Oren Eini just talks for about 40 minutes, answering pretty much just one or two questions.

Oren Eini on Building Projects that Endure by Technology and Friends (YouTube)

At one point (I forget where), he says,

“I don’t like unit tests.”

Agreed. I likelove automated tests. They’re indispensable. But I think unit tests are only useful when you want to focus on a failing integration test. David rightly points out that they’re really good for pinpointing where a problem actually happens, but Eini says that they also “hinder change” because, by their nature, they lock down a lot of the design and implementation.This is absolutely true.

Just to be clear: I think of anything that’s not a unit test as an integration test. I generally like “smaller” integration tests.

It’s probably better to just be agile about it and write them when the situation requires it, i.e., when the cause behind a failing integration test is proving difficult to pin down—or when you’ve determined the cause and you want a direct proof that you’ve fixed the underlying problem.

Ir requires discipline to realize when you need to write more unit tests in order to help pinpoint which component involved in a failing integration test is causing the problem. If you preemptively write all of the unit tests, you’re wasting time that could be better spent elsewhere.

I have had no small amount of success with a large test suite that was mostly integration tests. It ran relatively quickly (10 minutes for 10,000 tests on a reasonably classed developer desktop) and helped me survive three major refactorings.
 

The following video is a talk by Robert Martin “Uncle Bob”, one of the graybeards worth listening to. This video from 2011 is wide-ranging and contains a lot of brilliant advice. It’s stuff that we’ve known for a long time now, but every generation of programmers needs to re-learn these things about every 5-10 years. You usually can’t stop people from just reinventing the wheel because who wants to watch videos of or read blog posts written by old dudes, ammirite?

Ruby Midwest 2011 − Keynote: Architecture the Lost Years by Robert Martin by Robert C. Martin in 2011 (YouTube)

At 10:00, he talks about how the top-level architecture of most applications reflects the framework used to implement the web-delivery mechanism rather than the purpose of the application itself. In his example, he shows how a Ruby-on-Rails application is immediately recognizable as such, but that you have literally no idea what the application does.

He urges us to consider what this implies about our priorities as architects and developers. It means that we are much more concerned with the technology than with the functionality. This is not good.

He contrasts it with a high-level. 2-d blueprint of the first floor of a church, where the intent is obvious: it’s a church (he says). Of course, inferring that it’s a church involves applying the appearance of the diagram to a given context—e.g., a very western one—but the point is clear: the standard, top-level view of the design of a church screams out that it’s a church. It says nothing about how the church is to be built—or has been built—it says what it is.

“Architecture is about intent.”

Just to be clear: this presentation is from 12 years ago, and we’re still confronted with the same concepts—still confronted with the same failure to remember these precepts. Our frameworks still push themselves to the fore.

This is, in a way, the problem with LLM-generated code: we are already terrible at expressing the intent of our software in a way that makes it maintainable and qualitative. We are already mostly terrible at designing and building things in a way that satisfy the nearly-always-implicit non-functional requirements, like maintainability, usability, performance, etc.

And now we’re asking another piece of software, whose workings we can’t yet fathom, but which we know we’ve built by feeding it all of these terrible versions of software, and asking it to write software for us. All of the theory that we’ve developed about how to build software will not be respected, except by luck, if the neural net is feeling like that’s a high-probability next token.

On the one hand, I have to admit that this doesn’t sound much different from how software is built today, except that the human builders are potentially capable of following rules, whereas the software-based builders are less trainable. Again, though, we have decades of experience showing that, while people are ostensibly trainable, they are not necessarily practically trainable, at least in the general case for the general type of person who takes part in this field of endeavor we call programming.

Which leaves us with the question: have we achieved the maximum potential in software development? We already knew everything we needed to know about how to do it decades ago. What is missing is the will to do it that way. It’s definitely possible to train people to do it that way. The hangup is, as always, the cost, specifically, the cost-benefit ratio. The perceived benefit of better software is usually far less than the perceived (initial) cost.

And we always perceive only the initial cost because we are super-bad at long-term thinking about complex problems like building software.

At 34:00, Uncle Bob says

“There’s gotta be some better way to do this. […] This is just 3270 programming poisoned with all sorts of crud. How many languages do you have to do know to write a web application? Well, there’s some programming language, but that’s incidental! You’ve gotta know HTML and CSS and JS and Zazzle and Dazzle and … and, you know, the guy over here’s going: ‘let’s build communities by leveling people up. Leveling them up! I mean, what we’re going to do is hand them a … OK, now, hold this hammer. Ok? Good. You got that hammer? Now, here’s another one. Hold that hammer too. Now I’ve got a big barrel you’ve got to hold on your head. We are not helping our cause with this truly terrible mechanism that we have adopted.”

At 41:00, he says

“The database is a detail.”

This reminds me of The UI is an afterthought, a detail, an article I wrote recently [1] about a 7-year-old video I watched that expressed the same sentiments about external systems that Martin is expressing in his 12-year-old video.

“That’s what architecture is: find some place to draw a line and then make sure every dependency that crosses that line goes in the same direction.”

At 55:45, he says,

“There’s an interesting case of the database—the thing that’s so incredibly important—and yet, we took that decision and we just deferred it off the end of the world and then, when somebody needed it, we shimmed it in in a day. Because our architecture had done something right. What is the hallmark of a really good architecture? A good architecture allows major decisions to be deferred.”

“A good architecture maximizes the number of decisions not made.”

At 1:00:50, he says,

“How do you keep the beast under control? You need a suite of tests you trust with your life. You must never look at that suite of tests and think ‘you know? I don’t think I really tested everything?’ As soon as you think that, you’ve lost it. Because now you’re afraid of your code. The reason we write our tests first is so that we know, that every single line of code we wrote was because of a failing test that we wrote. So that we know that every single decision that we made is tested. So that then, we can pull up that code on our screen and say ‘Oh my God, that looks like a mess’—and clean it!…without any fear.”

Great talk. Add it to the pile of things that we know—or should know—better, but don’t.

As I was reading the absolute train wreck of a unit test in Testing with a Lisp (Daily WTF), the song “What the fuck is going on?” popped into my head, like it always does when I see that a programmer not only didn’t understand the assignment, not only doesn’t know how to program, but also doesn’t know that they don’t know how to program.

They are living their best life because they don’t think that “knowing how to program” is required in order to be a programmer. Neither does their boss or team, I guess.

That’s when the music starts to play in my head, and I think of little blind Dillon playing football because a very non-PC friend [1] sent me that video so many years ago.

Am I going to link the video? Of course I am. Because I’m a terrible person. [2]

BLIND KID PLAYING FOOTBALL WHAT THE FUCK IS GOING ON by RelentingPeach (YouTube)

And this is the test from the article above.

test("Returned objects arguments immutable (a b)", function() {
  var result = lispParser("(a b)");
  expect(3);
  ok(typeof(result) === 'object', "result is an object");
  var children = result.arguments;
  var newValue = 2;
  var firstChild = children[0];
  if (children[0] == newValue) {
    firstChild = ++newValue;
  }
  notEqual(result.arguments[0], newValue, "Underlying array was immutable");
  equal(result.arguments[0], firstChild, "Underlying array was immutable");
});

Highlight anything you think seems stupidNothing about that test makes any sense. It will always pass. It is, in its own way, a work of art. It is the JavaScript equivalent of Chomsky’s Colorless green ideas sleep furiously (Wikipedia), an example of a sentence that is “grammatically well-formed, but semantically nonsensical”.

Honestly, this looks worse than anything I’ve seen my students try to write. They usually have enough shame that they don’t bother filling in an answer if they really have no idea what’s going on.

I’m also wondering, of course, whether this is the work of an AI—or the bastard child of a poseur-programmer and an AI. The future is bright.

I was recently asked something like the following question, which I am citing with a few minor edits.

We would like to do a course about SW development with Python, preferably an online course, so that we can start at our own pace.

We don’t want a Python course, but would instead like a course more about SW development. It would be great if it were in Python because we are comfortable with it.

The interesting topics would be:

• object-oriented programming
• functional programming
• design patterns
• good coding practices

As well as other important topics such as:

• Testing
• Documenting
• Version control
• Working in a team with version control

The course doesn’t have to contain all these topics. It can be also several courses or it can be toy-projects from somewhere.

Learning how to develop software

I have very little familiarity with courses as I’ve usually been tasked with figuring out how to do things before others have gotten to it. Of late, I’ve been teaching courses, not taking them.

So, how did I learn what I know about software development? When I started writing software, there was nothing available online, outside of a bunch of GeoCities pages (one of which was mine). MSDN was on CDs or local help files.

I read some books, OOSC and OOSC2, as well as the Gang of Four’s Design Patterns. I can’t remember what else, but that’s partly how I leveled up my skills. I had the great fortune of being able to build and work on large frameworks, from which I drew many lessons. I worked with very good people, who challenged me and taught me a lot.

Nowadays, I use DuckDuckGo as my online reference. I have developed a relatively advanced skill at searching for what I’m looking for. I very often get it within minutes. I almost never use videos.

a primary skill in software development is to be able to imagine what you should be looking for. That is, you don’t have to know how to do everything without looking it up, but you do have to imagine that it might exist.

For example, I don’t know how to write automated tests in Python, but I know that it should be possible. I know that I should figure that out very early in my experiments with Python. I know what to expect from an automated-testing environment. I know which settings to look for and expect.

That kind of knowledge transfers from one language or development environment to another. I know that I code-completion makes me faster, I know that I would like to avoid runtime errors—how can I best use Python to achieve those ends?

Online Courses

I took a quick look around for online courses, but was not immediately convinced that I am equipped to be able to distinguish between scams and actually worthwhile courses. Does the course even mention general software-development principles? How much time is allocated to that?

Udemy

The Complete Software Engineering Course with Python (Udemy) looks as follows:

Udemy Python Software Course

• Only about 1% of students even bothered to rate the course
• The pricing fees high pressure and scammy
• The course descriptions are barely in English

What about general programming?

Udemy Software Engineering Lecture: 9.5 minutes

Just over nine minutes? And you can’t even be bothered to describe it in something approaching well-written English? No, thanks.

PluralSight

The course Learning To Program − Part 2: Abstractions (PluralSight) looks a bit more professional, but it still has some quirks (especially for $29 per month).

PluralSight Details

• Overall more professional than Udemy, but also seems to have been incompletely translated from French
• Still, there’s definitely more time dedicated to core concepts

There is an assessment that you can take, but you have to sign up first.

PluralSight Assessment

Maybe PluralSight is able to tell you which courses you need, but I doubt it will err on the “you need fewer courses” side.

Dometrain

I’ve recently heard from a source I’ve been watching for a while that this course is quite good for C# developers: From Zero to Hero: Test-Driven Development in C# by Guilherme Ferreira. The person recommending it releases quite interesting/advanced videos on YouTube and has his own range of courses at DomeTrain.

How should software-development be taught?

How would I teach basic software-development principles? I would probably start with very abstract principles that try to answer the classic questions for “use cases”:

• Requirements: What are you trying to achieve?
• Actors: Who is trying to achieve it?
• Stakeholders: Are there other points of view than just the actor’s?

Which language?

A question people tend to start with is: which programming language should I use?

That’s the wrong question.

The applicability of programming languages to fields differ widely, but most languages have a large overlap in functionality. Where they differ is in the degree of runtime or library support for specific tasks.

For example, Python famously has a lot of libraries for number-crunching and data-analysis (although I feel that this advantage is grossly exaggerated) whereas it’s terrible for writing Windows GUI applications. C#/.NET has excellent web and desktop technology support. The Python runtime is notoriously slow (with essential libraries written in C++) whereas .NET is known as a very performant cross-platform runtime.

Do you see how quickly the conversation turns from “what can the language do?” to “what can the standard runtime/libraries/environment do?” That’s because you can do most tasks with most languages.

Instead, we want to think about this at a higher level. We want to,

• maximize useful expressiveness while minimizing harmful expressiveness.
• accommodate inherent complexity without introducing accidental complexity.
• express our intent explicitly in our programs.
• be able to discover and eliminate assumptions
• get compilation errors or warnings, not runtime errors.

Developer discipline

Programming languages exist on several spectra. One of these is “the degree of developer discipline required to use the language effectively and safely.”

What does that mean? For example, Python and JavaScript have a dynamic type system. While there are mechanisms, practices, and IDE support that you can use to set up guardrails missing in the language, but they are optional and Idiomatically written code in both of these languages tends not to use any of it. It’s the wild west, for the most part, with a lot of assumptions that nothing will ever go wrong.

More strict languages force you to consider all possibilities before your program even compiles or runs. For example, Haskell and Rust are famously picky. If you have a function that returns a value under certain conditions, those languages will make you explicitly indicate what to return when those conditions don’t hold. Forgiving languages will just use some default value, usually null or undefined.

This is called “happy path” programming because you only write the code for the hoped-for path through your use case. For example, the user selects a valid file with the expected data format with an acceptable length with no validation or processing errors, generating a data file to which the initiating user has access.

• Did the user click cancel? Not handled.
• Was the file missing? Not handled.
• Was the file in an unexpected encoding? Not handled.
• Did the file fail to validate? Not handled.
• Was the data empty? Not handled.
• Did processing crash? Not handled.
• Was the generated output not accessible to the user? Not handled.

Writing programs in this fashion is a dangerous thing to do with a strict language, and it’s even worse to do in a lax language.

Even the simplest software has many, many branches. The less your language or compiler or IDE reminds you of them, the more you have to fill that gap with developer discipline.

Important language/runtime/IDE features

To get more concrete, some good questions to consider are:

• Can you clearly describe and use types? (implicitly typed ≥ explicitly typed ≥ dynamically typed)
• Are types statically checked? (yes > no)
• Can you primarily work with early binding? E.g., how strongly idiomatic are virtual/late bindings?
• Are data and operations idiomatically merged or separate? Is there support for pure data structures (e.g. records vs. classes)
• What about discriminated unions? Range types? Can you avoid primitive obsession?
• Can you declare non-nullable references?
• Can you designate functions as pure?
• Is a functional programming style supported?
• Is there a way to make data or records immutable?
• To what degree can you optimize performance where needed?
• How does error-handling work?
• How concise can you be? What kind of abstraction mechanisms are there? Do you have to write a lot of boilerplate?
• How does I/O work? Is it a second-class citizen? (I.e., does the language enforce purity so hard that it make it a pain to read from a file? I’m looking at you, Haskell and Elm)
• What is the asynchronous programming model?
• How good are the error messages?

If these don’t make any sense to you, don’t worry. But they are questions that are important when you’re choosing a tool for building software.

Intent & handling events

The whole point of a programming language is to express intent. You indicate what you intend to happen when a given event occurs.

An programmer expresses an intent by writing that, “when this thing happens, I intend for this other thing to happen.”

For example,

• When a filename is provided on the command line, read the contents of the file in a given encoding, process it as lines of text, and save the results to another file.
• When the user clicks the screen in a particular place when the program is in a particular state (e.g., over a button displayed in a dialog box), then execute an operation.
• When an HTTP request arrives, then read the body, process it, and return an HTTP response with the results
• When a sensor triggers an interrupt, then change a GUI readout from red to green
• When the system sends a shutdown message, then close and flush all logs

Questions to consider

How do we choose a programming language? You’re not just choosing a programming language, you’re also implicitly deciding which subset of language features to use. This is predicated, of course, on knowing about these features. It’s best to inform yourself about what your language/libraries/runtime (let’s call it a software-development tool) can do for you—or find someone who is well-informed to help.

For each feature, you should ask yourself: how useful is it? Does it help me achieve my task?

Let’s take a look at high-level features of a software-development tool that may be important.

Maintainability

• By whom? What level of programmer?
• How much will a certain part need to change?

Comprehensibility

• By whom? What level of programmer?
• Are you applying the “rule of least power”? That is, are you using the simplest, most effective tool in your arsenal? To know this, you have to expand your arsenal … and then use it judiciously
• Low syntax noise

Discoverability

• Can you use the IDE and code-completion to learn the API?
• How much documentation do you need?
• How much on-boarding would a new developer need?
• How easy would it be to hand off to another person or team?

Learnability

• Even if not especially discoverable, how learnable is the code?
• Are the patterns consistent?
• Is the API clear?

Strictness / Correctness

Error-free is strongly encouraged or enforced

Complexity

Inherent vs. Accidental

Idiomatic

Is it written in a way that other programmers of this language or framework can easily understand?

Testability

• Can you write semantically useful tests?
• Is it easier to write a test than to debug?

Debuggability

• Can your debugger set a breakpoint on all of the important bits?
• Can you even debug it?
• Is there too much magic?
• Is there so much generalization that you can’t figure out what’s going on?

Observability

• Is there sufficient logging to figure out what’s going on without debugging? (This is important once the product is in the field.)
• What about error-handling? Are problems separated from errors? [1]
• Can the developer predict what’s happening and measure it?
• How’s your telemetry?
• Could the software be monitored if needed?

Performance

• Does it need to be fast?
• All of it? Or just parts?

For code designed to be reusable (libraries, frameworks), you can also consider:

Completeness

degree to which the definition/API captures all facets of the problem domain

Expressiveness

Concise and precise

Flexibility

Applicability to different problem domains

Which of the features above matters more depends on what you’re building. A one-off script doesn’t need to satisfy many of these features. A full-blown application that needs to be maintained for 10-20 years by different teams has to be much, much more careful.

Other articles

This isn’t the first time I’ve written about these ideas, so I’ve included links to other, similar articles below.

General Programming Practices

These articles discuss the topic of software-development on a similar level to the discussion above.

• Elegant Code vs.(?) Clean Code
• Programming-language Features: How much is too much?
• Applying the rule of least power in practice
• Framework Design: Programming Paradigms and Idioms

The articles below are more recent, are more-or-less on the same level, but are also more targeted.

• How to evaluate dependencies
• To crash or not to crash; that is the question
• Stop trying so hard to use pattern-matching
• The UI is an afterthought, a detail
• On the usefulness of containers like Docker
• The “Hustle” culture in Software Development

Encodo White papers archive

These white papers were written from 2006 to 2019 when I was still employed at Encodo Systems AG. They expand on recommended practices of specific facets of software development. They are presented in reverse-chronological order, but can be read in any order.

• DI, IOC and Containers (2019)
• Clean and Safe Code (2019)
• Testing as First-Class Citizen (2019)
• Code Reviews (2017)
• Document Everything (2017)
• Continuous Integration and Delivery (2017)
• Component-based Design (2006)
• Design by Contract (2006)
• Test-driven Development (2006)

Videos

This is a YouTube playlist I’ve maintained for years that I continuously update whenever I watch a video that I think would be interesting for other developers. It’s only technology videos, but it’s pretty eclectic (i.e., it’s language- and technology-agnostic).

Developer suggestions (YouTube)

Conclusion

Pace yourself. You can’t have everything all at once. Programming takes wisdom. Wisdom takes time. It takes practice. It comes, or it doesn’t. It takes different forms.

As Rainer Maria Rilke wrote in 1903 [2],

“Forschen Sie jetzt nicht nach den Antworten, die Ihnen nicht gegeben werden können, weil Sie sie nicht leben könnten. Und es handelt sich darum, alles zu leben. Leben Sie jetzt die Fragen. Vielleicht leben Sie dann allmählich, ohne es zu merken, eines fernen Tages in die Antwort hinein.”

Good luck.

Note: I found this old draft containing my response to a colleague.

I 100% agree with you, in general. I absolutely want to know immediately when an assumption I’ve made does not hold.

But…😁

The degree to which I’m willing to crash depends on whose consistency I’m basing my assumptions on. When I call a method in my code from another method in my code, I’m absolutely going to assert that an argument is not null. I can control that. My IDE will tell me when I might be passing null. That is definitely a programming error.

When I’m getting external input (e.g. from the Windows registry), I’m a bit more cautious because I’m less sure about how solid my assumption is. I know what the documentation says but a lifetime of programming has taught me that some things (like the Windows registry) are going to work exactly as expected on my (modern) developer machine, but are going to fail mysteriously on a (perhaps less modern) machine in (for me) completely unpredictable ways.

Therefore, I’m a bit careful about is what I’m willing to pay to find errors. The primary purpose of a program is to bring value to the customer/user. I want to improve my program for more situations, but how am I going to find out in which situations it doesn’t work?

I can test, of course, but some things will only ever happen in the field. If it happens in the field, then I’m using the customer’s/user’s time to help me fix my program (they benefit, of course, but not for free). Can I soften the blow to the user of having to help me improve the program without sacrificing consistency or accuracy?

Sometimes, the answer is a resounding no. The program absolutely cannot continue if e.g., the reference to the data it needs to work on is null. That’s a no-go. There’s no rescuing the program from that or completing any other useful work.

In the case of this tool, if it crashes, the user no longer gets a report. Would they have been able to get some of the report if it hadn’t crashed? In this case, yes. All of the other checks could be run. The checks that crashed would show as “failed” with the exception message. That seems to me to be better than skipping all subsequent checks when one crashes.

I can even continue to hope that the user then reports the mysterious error message they got for one of the reports! Die Hoffnung stirbt zuletzt!

I’m delighted to discuss programming and error-handling philosophy in person next week!

This article is a copy of the white papers and process description that I wrote for Encodo Systems while I still worked there. I’ve preserved a copy of it here and in the linked articles.

Through our many years of experience building software, we’ve accumulated methodologies and principles that lead to quality software.

• The experience and know-how we bring to our consulting services also make us excellent partners in development-process consulting.
• We have experience in reviewing existing software and proposing changes to improve its reliability and stability.
• Not only can we build great software for you, we can help your development teams improve their current software and teach them how to write better software in the future.

Listed below are our methodologies.

DI, IOC and Containers (2019)

Applications are graphs of components, each with dependencies and inputs and outputs. DI and IOC are a simple and powerful way of building testable, flexible and easily replaced components.

Clean and Safe Code (2019)

A large part of writing good, clean code is to restrict your options. Avoid nullable data. Avoid mutable data. Avoid state. Code is not a poem: it’s actually preferable if it all looks the same and seems “easy” and “obvious”.

Testing as First-Class Citizen (2019)

Use tests to execute your code. Automate them to guard against future changes. Make testing easy for developers by building nice integration suites. Tests are code.

Code Reviews (2017)

Finished code is reviewed before it is committed. Reviews make sure that the other parts of the design process were followed (documentation, tests, etc.) and that errors are fixed as quickly as possible.

Document Everything (2017)

Software without documentation is not maintainable. Good naming conventions, logical designs and well-built tests help make code self-documenting. High-level tutorials explain how the different components are intended to work together.

Continuous Integration and Delivery (2017)

Low-overhead releases lead to faster turnaround and higher quality. Automated (or nightly) builds pinpoint software errors before they can become problems.

Component-based Design (2006)

Components encapsulate specifications. Software is designed to balance reuse and abstraction against the project goals, all the while optimizing maintainability.

Design by Contract (2006)

The specifications are built right into the code. Software publishes its assumptions instead of leaving its contract implicit.

Test-driven Development (2006)

Tests are a central part of building software. All component features have application-independent tests which can be run automatically and throughout development.

Encodo keeps the SOLID principles in mind when designing software.

DI & IOC

We implement the Inversion of Control (I) pattern with the dependency-injection pattern (D) to allow for a large amount of flexibility in how an application is composed. We’ve applied this principle throughout the Quino framework and use it in our products as well.

What does this mean? It means that the product or framework doesn’t make any decisions about which exact components to use. Instead, it indicates the API Surface (interface) that it expects in the form of injected components. That is, the responsibility for deciding which component to use lies not with the lowest level of the software stack, but with the highest level.

This inversion means that the application entry point configures the object graph (i.e. which objects will be used). That makes it much easier to isolate and test individual components, especially where those components would depend on native- or web-only functionality in production.

See the How do I DI? presentation from February 2018 for more information.

Principles

In order to make good use of this pattern, an application should adhere to the following rules:

• Prefers composition over inheritance, exposing clear dependencies.
• Refers to dependencies via interface or protocol types with as small a surface area as possible.
• Obtains dependencies through injection, preferably via the constructor.

Components built in this manner are agnostic in their implementation. They can be composed by an application as it sees fit.

Components

An application is a graph of components, each with one responsibility (S) and zero or more dependencies, injected via the constructor. Components are composed with other components to build higher-level functionality (O). They are also unaware of the other components’ implementations and can be replaced with other implementations (L).

Components make software flexible:

• Products can replace any component without changing anything else
• Products can inject any component without pulling in more functionality than needed

Declaration & Implementation

Components have a very clear purpose (S) indicated through an interface. In most cases, we use an actual “interface” language construct to clearly define the API surface and to not limit a product in its implementation (e.g. with an abstract base class).

• Prefer composition to inheritance, exposing clear dependencies
• Reference dependencies via interface (or type or protocol, depending on language) with as small a surface area as possible
• Obtain dependencies through injection, preferably in the constructor

Most components have a single method, amounting to a functional interface and allowing composition with lambdas. While TypeScript has this feature (as does Java), C# does not. We end up defining a lot of single-method classes that implement a single interface. It’s more code than we’d like, but it’s purely structural syntax and doesn’t introduce additional complexity.

See the Interfaces, base classes and virtual methods in the Quino conceptual documentation for more information and on and examples of patterns that we use.

Containers

Although it’s possible for applications to manually create an object graph (the composition root), we prefer to use an IOC Container.

The container provides two services:

• Registration: Applications declare the object or type and the lifetime (generally singleton) to use for interfaces
• Requests: Applications request objects, which the IOC creates—injecting other registered objects, as necessary—or retrieves, depending on lifetime. A container can create transient objects even for unregistered types.

The container introduces the following restriction:

• A concrete type may have many constructors, but only one may be public

The lifetime of an application is as follows:

• Collect registrations in the IOC Container
• Create composition root with the IOC Container
• Apply method to composition root

See the Quino Application Configuration for more information about application lifecycle. The blog article Starting up an application, in detail is a bit older, but provides more detail on how Quino integrates the IOC into the startup.