Investigate failing tests after bootstrap disabling

[surayya-MS]

#11865
#11866
#11658
After disabling bootstrap these tests are failing.

[MichalPavlik]

@GangWang01 could you please investigate why are there tests failing?

[GangWang01]

We need to get bootstrap back to work. After my investigation, the CI error before disabling bootstrap in the PRs was caused by dotnet/arcade#15894.

error MSB4242: (NETCORE_ENGINEERING_TELEMETRY=Restore) SDK Resolver Failure: "The SDK resolver "Microsoft.DotNet.MSBuildSdkResolver" failed while attempting to resolve the SDK "Microsoft.NET.Sdk". Exception: "System.IO.FileNotFoundException: Could not load file or assembly 'System.Text.Json, Version=8.0.0.4, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51' or one of its dependencies. The system cannot find the file specified.
File name: 'System.Text.Json, Version=8.0.0.4, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51'

[surayya-MS]

@GangWang01 , for more context , here it is okay for bootstrap to be disabled. It is expected. The strange thing is that tests depend on it. The goal is to understand why the tests depend on bootstrap and make them not to depend on it if possible.

[GangWang01]

@surayya-MS without bootstrap some assemblies were resolved with wrong path when running failed tests. It seems not easy to fix from the tests like Microsoft.Build.UnitTests.MSBuildTask_Tests.PropertyOverridesContainSemicolon that directly calls MSBuild API.

Here is the error of the failed test Microsoft.Build.UnitTests.MSBuildTask_Tests.PropertyOverridesContainSemicolon net472 of Windows Core.

Target "_HandlePackageFileConflicts" in file "D:\a\1\s\.dotnet\sdk\8.0.201\Microsoft\Microsoft.NET.Build.Extensions\Microsoft.NET.Build.Extensions.ConflictResolution.targets" from project "C:\Users\VssAdministrator\AppData\Local\Temp\oxea2whf.mfk\8211fce2c662425bbab97d0df3ead122\bug'533'369\Sub;Dir\ConsoleApplication1\ConsoleApplication1.csproj" (target "ResolveAssemblyReferences" depends on it):
D:\a\1\s\.dotnet\sdk\8.0.201\Microsoft\Microsoft.NET.Build.Extensions\Microsoft.NET.Build.Extensions.ConflictResolution.targets(30,5):  error MSB4062: The "ResolvePackageFileConflicts" task could not be loaded from the assembly D:\a\1\s\.dotnet\sdk\8.0.201\Microsoft\Microsoft.NET.Build.Extensions\\tools\net472\Microsoft.NET.Build.Extensions.Tasks.dll. Could not load file or assembly 'file:///D:\a\1\s\.dotnet\sdk\8.0.201\Microsoft\Microsoft.NET.Build.Extensions\tools\net472\Microsoft.NET.Build.Extensions.Tasks.dll' or one of its dependencies. The system cannot find the file specified. Confirm that the <UsingTask> declaration is correct, that the assembly and all its dependencies are available, and that the task contains a public class that implements Microsoft.Build.Framework.ITask.

Microsoft.NET.Build.Extensions.Tasks.dll actually exists under D:\a\1\s.dotnet\sdk\8.0.201\Microsoft\Microsoft.NET.Build.Extensions\tools\net8.0.

Getting bootstrap back and fixing the arcade issue mentioned above is easier to fix it. In addition, over time we will run into the same arcade issue for other release branches such as vs17.12.

[surayya-MS]

Thanks @GangWang01! We cannot change STJ version because of VS. In that case, if any of the skipped test depends on bootstrap and cannot be untangled from that, it's okay to leave it as skipped for these branches.

[GangWang01]

@surayya-MS it doesn't need to change STJ or other dependency version. Like #11951, it only gets the bootstrap back without any change to the dependency and re-enables the failed tests as well. Using specified VS and tool version in CI run fixed the problem.

[surayya-MS]

Oh, I see! Thanks, let's do that

[GangWang01]

If dotnet/arcade#15894 isn't resolved, alternatively we can use the images with old VS requested to FR in this conversation.

@GangWang01 is it still in blocked state?

[GangWang01]

@GangWang01 is it still in blocked state?
@ozanMSFT it's blocked by the arcade update that hasn't been flowed in for vs16.11. For vs17.8 and 17.10, they have the arcade update, and I will work on them first.