fix(KERNEL-GEMM-CPU-ELEM-X86WIDE): replace typed scalar-tail dereferences with LoadUnaligned - #3138
Merged
Merged
Conversation
…nces with LoadUnaligned The scalar tails in every elementwise CPU GEMM tier cast the weight pointer to const uint16_t* and dereference it directly. When the weight buffer starts at an odd byte address (which ElemRepackWeight can produce in production), UBSan catches the misaligned load and aborts. Replace each typed b[expr] and row[l] dereference with vt::LoadUnaligned<T>(ptr + expr), which uses memcpy and is always safe. Thirteen sites across four files: cpu_matmul_elem.cpp (portable/Neon/ SSE2), cpu_matmul_elem_avx2.cpp, cpu_matmul_elem_avx512.cpp, and cpu_matmul_elem_f16c.cpp. The vectorized loops already use unaligned- tolerant intrinsics (_mm_loadu_ps, vld1q_f32, etc), so only the scalar tails needed the fix. The regression test calls the kernel function pointers from ElemGemmTier() directly with a 1-byte-front-padded weight buffer, since Tensor::Contiguous copies the weight into an aligned buffer before the kernel sees it. Under UBSan the old code crashes; the new code passes. Closes #2908 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki]
mudler
force-pushed
the
fix/ci-sanitize-misaligned-load
branch
from
September 11, 2026 15:52
ea19033 to
5588293
Compare
mudler
added a commit
that referenced
this pull request
Sep 11, 2026
Fixes the windows-msvc and role-discipline failures that blocked all open PRs since 57220ba. The macos-metal-mlx, sanitize-cpu address, and sanitize-cpu thread failures were already fixed by #3136, #3138, and #3141. windows-msvc: kv_cache_utils.cpp used fopen with "re" mode which MSVC does not support; build-windows-release.ps1 needed a dumpbin path fix role-discipline: commit 96c5e47 reached main without a row/ branch. Advances the enforcement floor from e1b5df1 to 96c5e47, the third advance, forgiving that one unrepairable violation. Re-pins the floor test assertion in lockstep and adds the third-advance section to the spec intake-triage: the spec change surfaced two untriaged _intake records (#558, #664) whose issue numbers appear in the forgiven-commits table. Both are already fixed by the tree; triaged #558 to BENCH-ORACLE-PIN-RECONCILE and #664 to GATE-WINDOWS-PORTABILITY-TARGET-SCOPE so the agent-record gate passes Paths: product (src/vllm/v1/core/kv_cache_utils.cpp), script (scripts/build-windows-release.ps1), checker data (scripts/ci-enforcement-floor.txt), test (tests/scripts/test_check_commit_trailers.py), document (.agents/specs/ci-enforcement-floor.md), record (.agents/issues/BENCH-ORACLE-PIN-RECONCILE/ISSUE-GH-558.md, .agents/issues/GATE-WINDOWS-PORTABILITY-TARGET-SCOPE/ISSUE-GH-664.md) Closes #3135 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki]
mudler
added a commit
that referenced
this pull request
Sep 11, 2026
…ation Commit 96c5e47 reached main without a row/ branch, freezing the agent-record gate: every PR based on current main fails it. Advances the enforcement floor from e1b5df1 to 96c5e47, the third advance, forgiving that one unrepairable violation. Re-pins the floor test assertion in lockstep and adds the third-advance section to the spec. The spec change surfaced two untriaged _intake records (#558, #664) whose issue numbers appear in the forgiven-commits table. Both are already fixed by the tree; triaged #558 to BENCH-ORACLE-PIN-RECONCILE and #664 to GATE-WINDOWS-PORTABILITY-TARGET-SCOPE so the agent-record gate passes. The windows-msvc-cpu job fails at test_openai_api_server.exe (STATUS_STACK_BUFFER_OVERRUN) but is not a required check: PR #3097 merged while it was red. The macos, sanitize-address, and sanitize- thread failures named in #3135 were fixed by #3136, #3138, and #3141. This commit resolves the last blocking failure. Paths: checker data (scripts/ci-enforcement-floor.txt), test (tests/scripts/test_check_commit_trailers.py), document (.agents/specs/ci-enforcement-floor.md), record (.agents/issues/BENCH-ORACLE-PIN-RECONCILE/ISSUE-GH-558.md, .agents/issues/GATE-WINDOWS-PORTABILITY-TARGET-SCOPE/ISSUE-GH-664.md) Closes #3135 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki]
This was referenced Sep 11, 2026
Closed
mudler
added a commit
that referenced
this pull request
Sep 11, 2026
…reference in dots3_note_vision with LoadUnaligned `Dots3NoteVisionBlockCastFp8` casts borrowed safetensors bytes to `const uint16_t*` and dereferences the typed pointer at `src[idx]` (line 367). When `BorrowStTensorBytes` hands the weight at an odd byte address, the load is undefined behavior and the `sanitize-cpu (address,undefined)` lane aborts. Replace `src[idx]` with `vt::LoadUnaligned<uint16_t>(src + idx)`, the same `memcpy`-based load that #3138 applied to the CPU GEMM scalar tails and that this row applied to the three consumers in its original scope. No loaded value, shape, dtype, or behavior changes. This site was not enumerated in #2579 because the file did not exist in the tree at the #2579 base SHA (`aedad724c`). It was introduced by `c8dad3857` (2026-09-01) and matches the same `reinterpret_cast<const uint16_t*>(<OwnedTensor>.bytes.data())` shape. The spec lists it under `## Owed` as #3146. Red-before: `./build-sanitize/tests/test_dots3_note_vision` under `-fsanitize=address,undefined` reports `runtime error: load of misaligned address 0x... for type const short unsigned int, which requires 2 byte alignment` at `dots3_note_vision.cpp:367`. Green-after: the same binary passes all 20 test cases (21851 assertions) under the sanitizer with no runtime error. Closes #3146 Row: `FIX-UNALIGNED-CONSUMERS-2540` FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki]
mudler
added a commit
that referenced
this pull request
Sep 11, 2026
…reference in dots3_note_vision with LoadUnaligned `Dots3NoteVisionBlockCastFp8` casts borrowed safetensors bytes to `const uint16_t*` and dereferences the typed pointer at `src[idx]` (line 367). When `BorrowStTensorBytes` hands the weight at an odd byte address, the load is undefined behavior and the `sanitize-cpu (address,undefined)` lane aborts. Replace `src[idx]` with `vt::LoadUnaligned<uint16_t>(src + idx)`, the same `memcpy`-based load that #3138 applied to the CPU GEMM scalar tails and that this row applied to the three consumers in its original scope. No loaded value, shape, dtype, or behavior changes. This site was not enumerated in #2579 because the file did not exist in the tree at the #2579 base SHA (`aedad724c`). It was introduced by `c8dad3857` (2026-09-01) and matches the same `reinterpret_cast<const uint16_t*>(<OwnedTensor>.bytes.data())` shape. The spec lists it under `## Owed` as #3146. Red-before: `./build-sanitize/tests/test_dots3_note_vision` under `-fsanitize=address,undefined` reports `runtime error: load of misaligned address 0x... for type const short unsigned int, which requires 2 byte alignment` at `dots3_note_vision.cpp:367`. Green-after: the same binary passes all 20 test cases (21851 assertions) under the sanitizer with no runtime error. Closes #3146 Row: `FIX-UNALIGNED-CONSUMERS-2540` FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki]
localai-org-maint-bot
added a commit
that referenced
this pull request
Sep 12, 2026
…3142) Commit 96c5e47 reached main without a row/ branch, freezing the agent-record gate: every PR based on current main fails it. Advances the enforcement floor from e1b5df1 to 96c5e47, the third advance, forgiving that one unrepairable violation. Re-pins the floor test assertion in lockstep and adds the third-advance section to the spec. The spec change surfaced two untriaged _intake records (#558, #664) whose issue numbers appear in the forgiven-commits table. Both are already fixed by the tree; triaged #558 to BENCH-ORACLE-PIN-RECONCILE and #664 to GATE-WINDOWS-PORTABILITY-TARGET-SCOPE so the agent-record gate passes. The windows-msvc-cpu job fails at test_openai_api_server.exe (STATUS_STACK_BUFFER_OVERRUN) but is not a required check: PR #3097 merged while it was red. The macos, sanitize-address, and sanitize-thread failures named in #3135 were fixed by #3136, #3138, and #3141. This commit resolves the last blocking failure. Paths: checker data (scripts/ci-enforcement-floor.txt), test (tests/scripts/test_check_commit_trailers.py), document (.agents/specs/ci-enforcement-floor.md), record (.agents/issues/BENCH-ORACLE-PIN-RECONCILE/ISSUE-GH-558.md, .agents/issues/GATE-WINDOWS-PORTABILITY-TARGET-SCOPE/ISSUE-GH-664.md) Closes #3135 FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki] Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
localai-org-maint-bot
added a commit
that referenced
this pull request
Sep 12, 2026
…reference in dots3_note_vision with LoadUnaligned (#3148) Dots3NoteVisionBlockCastFp8 casts borrowed safetensors bytes to `const uint16_t*` and dereferences the typed pointer at `src[idx]` (line 367). When `BorrowStTensorBytes` hands the weight at an odd byte address, the load is undefined behavior and the `sanitize-cpu (address,undefined)` lane aborts. Replace `src[idx]` with `vt::LoadUnaligned<uint16_t>(src + idx)`, the same `memcpy`-based load that #3138 applied to the CPU GEMM scalar tails and that this row applied to the three consumers in its original scope. No loaded value, shape, dtype, or behavior changes. This site was not enumerated in #2579 because the file did not exist in the tree at the #2579 base SHA (`aedad724c`). It was introduced by `c8dad3857` (2026-09-01) and matches the same `reinterpret_cast<const uint16_t*>(<OwnedTensor>.bytes.data())` shape. The spec lists it under `## Owed` as #3146. Red-before: `./build-sanitize/tests/test_dots3_note_vision` under `-fsanitize=address,undefined` reports `runtime error: load of misaligned address 0x... for type const short unsigned int, which requires 2 byte alignment` at `dots3_note_vision.cpp:367`. Green-after: the same binary passes all 20 test cases (21851 assertions) under the sanitizer with no runtime error. Closes #3146 Row: `FIX-UNALIGNED-CONSUMERS-2540` FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:regolo/glm5.2 [maki] Co-authored-by: Ettore Di Giacinto <mudler@localai.io>
mudler
added a commit
that referenced
this pull request
Sep 14, 2026
…to completed Move ci-enforcement-floor.md to .agents/completed/ (per AGENTS.md, superseded specs move, they are not deleted) and fix the three dangling markdown links it caused in gate-on-main.md, issue-index.md, and the spec itself. Retire three issue records that the move orphaned: ISSUE-GH-1809 (the enforcement-floor row, no longer canonical), ISSUE-GH-3155 (owned by the spec Owed section, already CLOSED by PR #3138), and ISSUE-GH-296 (intake record referenced from test_agent_role.py, GitHub issue deleted). Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:glm5.2 [maki]
mudler
added a commit
that referenced
this pull request
Sep 14, 2026
…to completed Move ci-enforcement-floor.md to .agents/completed/ (per AGENTS.md, superseded specs move, they are not deleted) and fix the three dangling markdown links it caused in gate-on-main.md, issue-index.md, and the spec itself. Retire three issue records that the move orphaned: ISSUE-GH-1809 (the enforcement-floor row, no longer canonical), ISSUE-GH-3155 (owned by the spec Owed section, already CLOSED by PR #3138), and ISSUE-GH-296 (intake record referenced from test_agent_role.py, GitHub issue deleted). FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:glm5.2 [maki]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The scalar tails in every elementwise CPU GEMM tier cast the weight pointer to
const uint16_t*and dereference it directly. When the weight buffer starts at an odd byte address (whichElemRepackWeightcan produce in production), UBSan catches the misaligned load and aborts the sanitize-cpu lane.Replace each typed
b[expr]androw[l]dereference withvt::LoadUnaligned<T>(ptr + expr), which usesmemcpyand is always safe. Thirteen sites across four files:cpu_matmul_elem.cpp(portable/Neon/SSE2),cpu_matmul_elem_avx2.cpp,cpu_matmul_elem_avx512.cpp, andcpu_matmul_elem_f16c.cpp. The vectorized loops already use unaligned-tolerant intrinsics (_mm_loadu_ps,vld1q_f32, etc), so only the scalar tails needed the fix.The regression test calls the kernel function pointers from
ElemGemmTier()directly with a 1-byte-front-padded weight buffer, sinceTensor::Contiguouscopies the weight into an aligned buffer before the kernel sees it. Under UBSan the old code crashes; the new code passes.Red-before:
VT_CPU_MATMUL_TIER=portable ./build-sanitize/tests/test_ops_matmul_elemaborts withruntime error: load of misaligned address ... for type const T, which requires 2 byte alignmentatcpu_matmul_elem.cpp:63.Green-after: all 8 test cases pass under UBSan with both the portable and avx512 tiers, and in a regular (non-sanitized) build.
Closes #2908
Row:
KERNEL-GEMM-CPU-ELEM-X86WIDEFOLLOWING_AGENTS_PROTOCOL
Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: AGENT:regolo/glm5.2 [maki]