Skip to content

JIT: Materialize partially-constant Vector.Create as CNS_VEC + inserts#130717

Open
tannergooding wants to merge 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-optimize-hwintrinsic-create-constants
Open

JIT: Materialize partially-constant Vector.Create as CNS_VEC + inserts#130717
tannergooding wants to merge 2 commits into
dotnet:mainfrom
tannergooding:tannergooding-optimize-hwintrinsic-create-constants

Conversation

@tannergooding

Copy link
Copy Markdown
Member

LowerHWIntrinsicCreate previously materialized a partially-constant Vector.Create as a chain of inserts starting from CreateScalarUnsafe, even for the elements that were constant. For example, Vector128.Create(1, 2, 3, x) produced four inserts rather than a single constant plus one insert for x.

IsHWIntrinsicCreateConstant now optionally reports which elements are constant via a simdmask_t. When two or more operands are non-zero constants, lowering materializes them as a single CNS_VEC and inserts only the remaining non-constant operands via WithElement. So Vector128.Create(1, 2, 3, x) becomes a CNS_VEC plus a single WithElement.

Only non-zero constants are counted towards the threshold. Zero lanes are already produced for free by the existing insert path (insertps zero-masking, CreateScalarUnsafe zero-extension), so materializing them into a constant would add instructions rather than remove them -- for example Vector4.Create(a, b, 0, 0) stays as two insertps and does not regress.

Larger vectors are still split into per-128-bit-lane Create nodes before reaching the base case, so this applies per lane and a fully constant lane collapses to a CNS_VEC combined via WithUpper/WithLower.


SuperPMI asmdiffs (windows x64, 2,596,354 contexts): -806 bytes overall, no regressions.

Collection Diff
benchmarks.run_pgo -327
coreclr_tests.run -456
libraries.crossgen2 -12
libraries_tests_no_tiered_compilation -11

Notable improvements:

  • System.Buffers.Text.Base64Helper:Avx2Encode -290 (-25.33%)
  • GitHub_43569.Program:Vector128_Create_byte -155 (-42.12%)
  • GitHub_43569.Program:Vector256_Create_float / Vector128_Create_float -21 each

Correctness was validated across all base types and vector sizes with non-constant elements at various positions, under Checked asserts, across ISA configs (AVX512/AVX2/AVX off) and JitStress=2.

Note

This PR description was drafted with the assistance of GitHub Copilot.

IsHWIntrinsicCreateConstant now optionally reports which elements are constant via a simdmask_t. When two or more operands of a Vector.Create are non-zero constants, lowering materializes them as a single vector constant and inserts the remaining non-constant operands via WithElement, rather than emitting a chain of inserts starting from CreateScalarUnsafe.

Only non-zero constants are counted towards this threshold: zero lanes are already produced for free by the existing insert path (insertps zero-masking, CreateScalarUnsafe zero-extension), so materializing them into a constant would add instructions rather than remove them.

Larger vectors are still split into per-128-bit-lane Create nodes before reaching the base case, so the optimization applies per lane and a fully constant lane collapses to a CNS_VEC combined via WithUpper/WithLower.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 14, 2026 18:41
@github-actions github-actions Bot added the area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI label Jul 14, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 5 pipeline(s).
10 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

@dotnet-policy-service

Copy link
Copy Markdown
Contributor

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves JIT lowering for partially-constant Vector*.Create(...) by materializing multiple non-zero constant lanes as a single vector constant (CNS_VEC) and inserting only the remaining non-constant lanes, reducing node count and typically improving codegen.

Changes:

  • Extend GenTreeVecCon::IsHWIntrinsicCreateConstant to optionally return a per-element constant mask (simdmask_t).
  • Add a shared lowering path (LowerHWIntrinsicCreateWithInserts) that builds a CNS_VEC and fills non-constant operands via WithElement/platform insert intrinsics.
  • Add a profitability helper (NonZeroConstantElementCount) and wire it into xarch/armarch LowerHWIntrinsicCreate for 128-bit lanes.
Show a summary per file
File Description
src/coreclr/jit/lowerxarch.cpp Uses the new constant-mask + non-zero-constant threshold to prefer CNS_VEC + WithElement over insert chains for partially-constant creates.
src/coreclr/jit/lowerarmarch.cpp Same as xarch: enables the new partial-constant materialization path on ARM64 lowering.
src/coreclr/jit/lower.h Declares the new helper lowering method and profitability helper.
src/coreclr/jit/lower.cpp Implements NonZeroConstantElementCount and LowerHWIntrinsicCreateWithInserts shared lowering logic.
src/coreclr/jit/gentree.h Extends IsHWIntrinsicCreateConstant to optionally report which lanes were constant via simdmask_t.

Copilot's findings

  • Files reviewed: 5/5 changed files
  • Comments generated: 5

Comment thread src/coreclr/jit/lower.cpp Outdated
Comment thread src/coreclr/jit/lower.cpp
Comment thread src/coreclr/jit/lower.cpp Outdated
Comment thread src/coreclr/jit/lowerxarch.cpp Outdated
Comment thread src/coreclr/jit/lowerarmarch.cpp Outdated
The helper checks the raw bytes so that -0.0 (sign bit set) is counted as non-zero, since the free zeroing paths would otherwise turn it into +0.0. Reword the comments to say all-bits-zero rather than non-zero so the intent is clear and not simplified to a numeric == 0 check.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 14, 2026 19:31

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot's findings

  • Files reviewed: 5/5 changed files
  • Comments generated: 1

Comment thread src/coreclr/jit/lower.cpp
@tannergooding

Copy link
Copy Markdown
Member Author

@MihuBot -nuget

@tannergooding

Copy link
Copy Markdown
Member Author

CC. @dotnet/jit-contrib, @EgorBo. This one should be ready for review.

Not as many hits as I would've liked, likely because the pattern isn't overfly common, but a few in ImageSharp and the numerics types. Also improves Base64Encode under PGO

Namely this just recognizes the Vector128.Create(cns, cns, x, cns) pattern (and similar) optimizing it to not always do 4 inserts and instead to lower it to vcns.WithElement(3, x). Pays off more for smaller T like byte and functionally matches what MSVC/LLVM do for similar

Code is centralized and shared across Arm64 and Xarch to avoid duplication, so we can easily hook it up for WASM next as well.

@MichalPetryka

Copy link
Copy Markdown
Contributor

Not as many hits as I would've liked, likely because the pattern isn't overfly common, but a few in ImageSharp and the numerics types.

I explicitly avoided it in the past in favour of const + insert in the past cuz it wasn't handled.

@tannergooding

Copy link
Copy Markdown
Member Author

Right, its what I've recommended and I imagine most would've done if they profiled due to lack of handling

So code can be cleaned up now if this goes in.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants