Skip to content

fix(mocks): reference TUnit.Mocks once in snapshot test compilations - #6506

Merged
thomhurst merged 2 commits into
mainfrom
fix/mock-snapshot-diff-output
Jul 28, 2026
Merged

fix(mocks): reference TUnit.Mocks once in snapshot test compilations#6506
thomhurst merged 2 commits into
mainfrom
fix/mock-snapshot-diff-output

Conversation

@thomhurst

@thomhurst thomhurst commented Jul 28, 2026

Copy link
Copy Markdown
Owner

Root cause

TUnit.Mocks.SourceGenerator.Tests handed each test compilation two TUnit.Mocks references: one from the AppDomain assembly sweep in SnapshotTestBase.LoadReferences, and one from a ref\TUnit.Mocks.dll copy of the netstandard2.0 build staged by the csproj.

Unless those two files are byte-identical, Mock and GenerateMockAttribute bind to an error type. Nothing reports it — not Roslyn (the harness never inspected the input compilation's diagnostics) and not the generator (no mock targets found is not an error). The generator emits only its post-init TUnit.Mocks.Generated namespace stub, so the test fails as a bare snapshot mismatch with a five-line received file.

The copy came from a <None Include> glob expanded at evaluation time, so whether it shipped at all — and which build it came from — depended on build ordering. That is why GenerateMock_Attribute_With_Concrete_Class and KiotaIRequestAdapter_ConstrainedGenericMethod_EmitsDefaultConstraint failed on the windows and macos jobs on every branch rebased onto current main (including #6501, which touches no snapshot file at all), while ubuntu and local Windows runs stayed green. main doesn't run the pipeline on push, so it was invisible there.

Fix

  • LoadReferences now references the TUnit.Mocks assembly the test project already loads, and excludes that same assembly from the AppDomain sweep — exactly one reference, no ordering dependence.
  • Dropped the ref\TUnit.Mocks.dll copy and the netstandard2.0 ProjectReference that existed only to feed it.
  • Snapshot mismatches now print the first divergence inline (line counts, line number, three lines of context, -/+ capped at 40). Without this the CI-only failure was undiagnosable from the log, since .received.txt never leaves the runner.

Verification

Planted a version-skewed TUnit.Mocks.dll in the old ref location to force the duplicate-reference state:

  • before the fix: 8+ tests fail, each with a five-line received file — reproducing the CI symptom exactly;
  • after the fix: 84/84 pass with that same skewed dll still sitting there.

Full suite green locally on net10.0.

A mock snapshot mismatch reported only the two file paths. That is
enough locally, but a mismatch that only reproduces on CI — currently
GenerateMock_Attribute_With_Concrete_Class and
KiotaIRequestAdapter_ConstrainedGenericMethod_EmitsDefaultConstraint on
the windows and macos jobs, both green on ubuntu and on a local windows
run — leaves nothing in the log to diagnose from, because the
.received.txt never leaves the runner.

Print the first divergence with three lines of context and a 40-line
cap, plus both line counts.
Comment thread tests/TUnit.Mocks.SourceGenerator.Tests/SnapshotTestBase.cs
@greptile-apps

greptile-apps Bot commented Jul 28, 2026

Copy link
Copy Markdown

Greptile Summary

This PR revises mock-generator test references and adds inline snapshot-difference diagnostics.

  • Uses the loaded TUnit.Mocks assembly as the single compilation reference.
  • Removes staging of the separate netstandard2.0 reference assembly.
  • Reports line counts, leading context, and differing lines when snapshots mismatch.

Confidence Score: 4/5

The PR appears safe to merge, although the previously reported diagnostic output-cap issue remains.

Sparse snapshot differences can still emit every intervening unchanged line because those lines do not advance the report counter, leaving the intended diagnostic bound ineffective.

Files Needing Attention: tests/TUnit.Mocks.SourceGenerator.Tests/SnapshotTestBase.cs

Important Files Changed

Filename Overview
tests/TUnit.Mocks.SourceGenerator.Tests/SnapshotTestBase.cs Uses one loaded mocks assembly for test compilations and adds snapshot mismatch diagnostics.
tests/TUnit.Mocks.SourceGenerator.Tests/TUnit.Mocks.SourceGenerator.Tests.csproj Removes the additional netstandard2.0 project build and copied reference assembly.

Reviews (2): Last reviewed commit: "test(mocks): reference TUnit.Mocks exact..." | Re-trigger Greptile

The test compilations were handed TUnit.Mocks twice: once from the
AppDomain assembly list and once from a ref\TUnit.Mocks.dll copy of the
netstandard2.0 build. Unless the two are byte-identical, Mock and
GenerateMockAttribute bind to an error type, the generator finds no mock
targets, and it emits only its post-init namespace stub — with no
diagnostic from Roslyn or the generator. The affected tests fail as a
bare snapshot mismatch whose received file is five lines long.

That is what broke GenerateMock_Attribute_With_Concrete_Class and
KiotaIRequestAdapter_ConstrainedGenericMethod_EmitsDefaultConstraint on
the windows and macos jobs while ubuntu stayed green: the copy came from
a <None Include> glob expanded at evaluation time, so whether it shipped
at all — and which build it came from — depended on build ordering.

Reference the already-loaded assembly and exclude it from the AppDomain
sweep, and drop the ref copy and its netstandard2.0 ProjectReference.
Verified by planting a version-skewed TUnit.Mocks.dll in the old ref
location: before, 8+ tests fail with five-line output; after, 84/84 pass
with it still present.
@thomhurst thomhurst changed the title test(mocks): diagnose the CI-only mock snapshot failures fix(mocks): reference TUnit.Mocks once in snapshot test compilations Jul 28, 2026

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review

The core fix is solid: excluding the loaded TUnit.Mocks assembly from the AppDomain sweep and adding it back exactly once (SnapshotTestBase.cs:41-47) directly closes the double-reference hole, and dropping the ref\TUnit.Mocks.dll copy + its SetTargetFramework=netstandard2.0 ProjectReference from the csproj removes the build-order dependency entirely rather than papering over it. Good root-cause fix, and the verification method described (planting a version-skewed dll) is a convincing way to prove it.

Bug: DescribeDifference's 40-line cap doesn't bound total output

In the reporting loop (SnapshotTestBase.cs:347-370), reported is only incremented when a line actually differs:

if (string.Equals(verifiedLine, receivedLine, StringComparison.Ordinal))
{
    report.Append("  ").Append(verifiedLine).Append('\n');
    continue;   // <-- reported not incremented
}
...
reported++;

For the most common real-world case — a single line changed with the surrounding file otherwise identical and the same total line count — every line after the one difference re-matches by position, so the loop keeps appending matched lines all the way to EOF without ever hitting the MaxReportedLines (40) cutoff. On a several-hundred/thousand-line generated snapshot, one differing line near the top would dump nearly the entire file into the failure message — exactly the CI log noise this change is meant to avoid (per the PR description: "capped at 40").

The cap needs to bound total lines emitted in this loop, not just the count of differing positions, e.g.:

for (var i = firstDifference; i < Math.Max(verifiedLines.Length, receivedLines.Length) && reported < MaxReportedLines; i++)
{
    ...
    reported++; // increment unconditionally, whether the line matched or differed
}

(or track a separate linesEmitted counter for the loop bound while keeping reported for the "... (truncated)" check if you want it to reflect only genuine diffs).

Everything else — the LoadReferences change, the csproj cleanup, the inline diff context — looks correct and well-reasoned.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant