I published a very similar version of the following article in the DevOps Wiki at Uster Technologies AG. Since nearly all of that post is general knowledge that I would have been happy to find before I started my investigations, I’m sharing it here.
When we think about navigating or debugging our code, we usually focus on the code we’ve written ourselves—local sources in our file system. IDEs have classically focused on being able to debug and navigate this code.
More and more, though, we’re also interested in navigating and debugging our versioned and compiled dependencies:
Most of these are available as source code. We would ideally like to be able to navigate and debug that code just as easily as we can our own.
The following sections define file types and terminology, and then explain how these concepts apply to debugging and navigation for external sources. You can also just jump to the sections on producing or consuming packages (especially as relates to authentication for private sources).
The following diagram provides an overview of the process of obtaining external packages, along with their symbols and source files. It looks quite complicated, but accommodates the flexibility required by various stakeholders.
There are several types of files associated with debugging and navigation:
DLL
PDB
XML
*.cs
It’s reasonable to ask why this process is so complex.
nupkg
just include the PDB
and the *.cs
files?The system was designed for use cases where most sources were closed. That has changed, but the system still reflects the original design choices. The PDB files can also add about 30% to the size of the package. The original use cases preferred to avoid using 30% more space for package downloads that didn’t need the debugging information.
Again, historically, the use cases were for providing improved stack traces with symbols, but not to provide access to closed sources. Even if the sources are partially open, access may be restricted to only some users of the packages or symbols. Having the IDE request the sources separately allows an additional authorization phase.
The defaults still reflect the original use cases, which actually represent fewer and fewer packages as time goes on.
These answers aren’t particularly satisfying if your use case happens to be “make a package that has symbols for excellent stack traces and sources for excellent debugging”. At least we now have IDEs that know how to work with this system and there is a lot of automation for producing packages with the desired symbol and source-code support.
A developer debugs source code by interrupting execution of a program—either manually or by setting breakpoints—and then stepping through the instructions, examining the contents of symbols (variables) to investigate the runtime behavior and operation of the system.
The debugger uses the PDB
to allow source-level debugging, i.e. debugging in the original source code. While debugging in “lower” formats is possible, it’s not nearly as reliable as being able to step through the code in the original source code, using the original symbols.
How does the debugger obtain the PDB
for a given DLL
?
DLLs
and PDBs
have unique identifiers that make it possible to request and download the correct file.Once the debugger has the PDB, it has everything it needs—except the source code.
If the PDB was generated locally, then it most likely references the source files that are still in the same locations in the file system as when it was built. In that case, the debugger easily finds the source files because they’re just at the paths that are directly referenced by the PDB
.
If the PDB was not generated locally or the source-code paths do not match, then there are other tricks to find the source files. Visual Studio allows you to set “Directories containing source code” for the “Debug Source Files”
If the sources aren’t available locally, e.g., for a NuGet package, then there is a system called SourceLink that is extremely well-supported in the .NET world that makes it possible to easily download the source files that generated a DLL
and that are referenced by its PDB
.
Things to be aware of:
If the package does not support SourceLink, but the sources are available, then you can download the sources locally and use the solution-level mapping above to tell the debugger where the source files are. You can also just point the debugger to the top-level folder when it asks for the file’s location, in which case the debugger makes the entry for you.
A developer navigates by requesting the source code for a symbol. For example, if the declared type of a variable in an open source file is the class Setting
, then the developer can ask the IDE to show the source of Setting
by Ctrl + clicking, by pressing F12 in Visual Studio, or by pressing Ctrl + B in Rider.
As with debugging, navigating local sources is straightforward, since the sources are in the local file system. For symbols in NuGet packages, the IDE has to be clever enough to download, cache, and use the sources.
Visual Studio on its own does not support navigating to external sources via SourceLink. Instead, it always decompiles external sources, as shown in the example below.
If you have ReSharper installed, then the default setting is to try as hard as possible to avoid showing a decompiled version.
You can also add “Folder Substitutions” in the “Advanced Symbol options…” for navigating to “External Sources”. The option does not seem to be available in Rider.
SourceLink is a system that provides source files for external sources like NuGet packages for debugging or navigation. In order for this to work, you must be able to provide external sources or the client is not properly configured for debugging.
See below for troubleshooting information, especially as relates to authentication for packages and source code pulled from authenticated locations.
A decompiled version of the source code is a reconstruction of the original source from the instructions and information in the DLL
and PDB
. When sources cannot be located for a given symbol, Visual Studio, ReSharper, and Rider will produce a decompiled version as a fallback.
This is often good enough to be able to read the code reasonably well, but it leaves certain common constructs in their “lowered” format. E.g., calls to extension methods appear as static-method calls rather than as targeted on the first parameter.
This can make debugging difficult, as the instructions don’t match the mapping. Rider has support for patching the PDB on-the-fly to allow more comfortable debugging of decompiled sources. This is, however, a fallback solution for external packages over which you have no control. It’s best to configure your packages to publish with symbols and sources available to IDEs that support them, as shown in the next section.
The documentation to Enable debugging and diagnostics with Source Link is thorough and tells you all you need to know about all of the options.
If you’re working with Azure DevOps Services, you should include the following package reference:
<ItemGroup>
<PackageReference Include="Microsoft.SourceLink.AzureRepos.Git" Version="8.0.0" PrivateAssets="All"/>
</ItemGroup>
With this, you’re all set. The package is published to the Azure Artifacts, with a corresponding snupkg
available on the Azure symbol server and sources available via the repository URL (subject to authorization; see below for troubleshooting).
You can set a few optional properties, detailed below. Most projects won’t need to set these, but they are included to spare you the research if you see them in code examples, either in your institution’s code or online. As noted, the only line you need is the package reference shown above.
IncludeSymbols
DebugType
is set to embedded
) or in a separate symbol package (if SymbolPackageFormat
is set to snupkg
). This is implied when the NuGet package Microsoft.SourceLink.AzureRepos.Git is included, as shown below.snupkg
when the NuGet package Microsoft.SourceLink.AzureRepos.Git is included, as shown below.See the SourceLink documentation for more details. Among other details, they also note that projects that target .NET 8 no longer need to include this support explicitly because Azure Repos are supported by default, as detailed in the readme for the SourceLink project.
“If your project uses .NET SDK 8+ and is hosted by the above providers (GitHub, Azure Repos, GitLab, BitBucket) it does not need to reference any Source Link packages or set any build properties.”
You can also include the packaging conditionally in the Directory.Build.Targets
, as shown below.
<ItemGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<PackageReference Include="Microsoft.SourceLink.AzureRepos.Git" Version="8.0.0" PrivateAssets="All"/>
</ItemGroup>
See the appendix for Directory.Build.Props
and Directory.Build.Targets
for more information about which variables and directives are respected in which file.
If a package has SourceLink enabled and you have access to the online repository from which it was built, then to seamlessly debug into that source code, ensure the following:
As noted above, Visual Studio doesn’t support navigating via SourceLink. To browse external sources with JetBrains tools, ensure the following:
Once you’re sure that the package supports SourceLink, then you should also make sure that the Just My Code setting is disabled.
When Just My Code is enabled, the debugger skips over any code that doesn’t correspond to source code in one of the local projects.
.pdb
file next to the .dll
file)?PDB
is not included with the package, is it available on a Symbol Server?DLL
?If it’s available in the package, but is not being copied to the output folder, then if you’re using .NET 7.0 SDK or higher, you can use the build property named CopyDebugSymbolFilesFromPackages.
<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<CopyDebugSymbolFilesFromPackages>true</CopyDebugSymbolFilesFromPackages>
</PropertyGroup>
Verify that the symbols for the module you’re trying to debug have been loaded. If they aren’t loaded, you can try to load symbols while debugging. For more details and a screenshot, see Just My Code debugging.
If you’re trying to navigate in code, but ReSharper or Rider keeps decompiling instead of getting the sources from SourceLink, then check your External Sources settings in ReSharper or Rider. Verify that the tool is configured to check for external sources before it tries decompiling.
If the IDE is having trouble authenticating, then you will usually see a decompiled version instead. Sometimes the code is so close to the original that it’s hard to tell; scroll to the top to see if it includes the “decompiled by JetBrains…” header.
Once the IDE has decompiled a source file, it will continue to use this cached copy until you close the tab, or sometimes you have to close and re-open the project. If you’re troubleshooting your way through this setup, then you can temporarily disable decompilation as a fallback, which avoids producing the unwanted source-code variant in the first place.
Visual Studio uses the authentication associated with the logged-in user that you use to enable the IDE. This can be in a weird state if you’ve recently changed your password or your authentication token is stale or in a non-refreshable state. Try logging out and back in.
JetBrains tools (Rider, ReSharper, DotPeek, etc.), on the other hand, need to be given a token.
If the tool shows a notification indicating that authentication has failed, then do the following:
Configure
on the notification to show a dialog
john.doe@example.com
Test
button to verify that it works (you should see OK 200
)Ok
to save the credentialsHowever, there is a bug whereby JetBrains tools fail to show a notification or offer a way to enter credentials.[1] That’s going to look something like this:
It claims that it can download the source, but it never completes. You have to cancel the dialog. If you then look at the ReSharper Output, then you’ll see something like this:
The relevant text is at the end of the third line, which indicates that the request for the source file returned a “Non-OK HTTP status code”.
PdbNavigator: Searching for 'Example.Core.AppConfig.AppConfigKeyAttribute' type sources in C:\Users\john.doe\.nuget\packages\example.core.appconfig\4.1.0\lib\netstandard2.0\Example.Core.AppConfig.pdb
PdbNavigator: File names (1) are inferred for type Example.Core.AppConfig.AppConfigKeyAttribute
PdbNavigator: Downloader: https://dev.azure.com/example/example.Core/_apis/git/repositories/Example.Core.LabInstruments/items?api-version=1.0&versionType=commit&version=8b34c2aa672facd47e835c27152f695fa796a408&path=/Example.Core/DotNetStandard/Example.Core.AppConfig/AppConfigKeyAttribute.cs -> Non-OK HTTP status code
The most reliable way to fix this is to create the credentials in the Credential Manager. Be aware that you will need to create an Azure PAT (personal access token).
Windows Credentials
JetBrains SourceLink https://dev.azure.com/exampleOrganization
If you don’t have this entry, then that’s the problem. If you have it, but you still can’t get the sources, then edit the entry to have valid credentials.
To create or edit the record, do the following from the Credentials Manager:
JetBrains SourceLink https://dev.azure.com/exampleOrganization
john.doe@example.com
As you can see above, although publishing a package is relatively straightforward, there are quite a few stumbling blocks on the way to consuming the package for navigation and debugging. Once you have everything set up and working, it’s great, but … there is still one other drawback.
You can’t edit the code for packages.
This is not optimal. Optimally, we’d like to quickly verify that change to an upstream code would address an issue in downstream code without having to generate new packages. It would be great to just edit the upstream code as if it were part of your downstream solution until you’re sure that the change would address your downstream issue. At that point, you can copy the changes back to the upstream solution (where the dependency is produced), add tests, and produce a new version, being pretty certain that the change is effective.
The shortest possible developer-feedback loop with code in external packages is:
PDB
)If your package has dependencies or your change in the external package’s solution touches multiple packages, then you can do the following:
If it get too complicated to do locally, then you can always commit, push, and have the CI generate new versions of your packages (hopefully with a prerelease version, e.g., 3.2.4-preview2
)
The solutions outlined above have a reasonable turnaround time, but sometimes you want to pretend that the external packages are just internal projects instead. This basically entails:
At that point, you can edit, debug, and navigate the code as if it were your own.
See the “Project Munging with Tools & PowerShell” section of How to Debug NuGet Packages with Symbols and Source Link Painlessly for a PowerShell script that can help you automate part of this.
MSBuild supports including common configuration in project files. While earlier versions required all configuration to be included explicitly, modern versions include configuration files with special names automatically, greatly simplifying common configuration and reducing clutter in project files.
If the file is named Directory.Build.Props
or Directory.Build.Targets
, it is picked up automatically and included for all projects in that folder or any subfolder. If you use a different name, then you have to explicitly reference that file from a project or from another *.props
or *.targets
file. If you choose your own name, you don’t have to use the Build.Properties
or Build.Targets
convention, but it’s strongly recommended, to avoid confusion.
You can use a Directory.Build.Properties file to include settings for all projects in a folder or set of subfolders.
For example, the following package reference can and should be included in Directory.Build.Props
:
<PackageReference Include="Microsoft.SourceLink.AzureRepos.Git" Version="8.0.0" PrivateAssets="All"/>
If you want to include settings conditionally based on build configuration (e.g., Configuration
or Platform
), then you’ll have to use the Directory.Build.Targets file, which has access to those variables.
<ItemGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
<PackageReference Include="Microsoft.SourceLink.AzureRepos.Git" Version="8.0.0" PrivateAssets="All"/>
</ItemGroup>
Directory.Build.Props
file at the root of the solution.