fix(kt-kernel): pick default CUDA archs by nvcc version (fixes #2098)

[Anai-Guo]

Problem

kt-kernel/setup.py sets -DCMAKE_CUDA_ARCHITECTURES=80;86;89;90 whenever CPUINFER_CUDA_ARCHS is unset:

archs_env = os.environ.get("CPUINFER_CUDA_ARCHS", "80;86;89;90").strip()

An architecture only becomes valid to nvcc in a specific CUDA toolkit release — sm_89 (Ada) needs CUDA 11.8+ and sm_90 (Hopper) needs CUDA 12.0+. On an older toolkit nvcc rejects the unknown -gencode targets, so the wheel build fails during compilation with no clear cause. This is exactly what #2098 hits on CUDA 11.5 (a 4×A100 box):

error: subprocess-exited-with-error
× Building wheel for kt-kernel (pyproject.toml) did not run successfully.

Fix

When CPUINFER_CUDA_ARCHS is not set, derive the default from the detected nvcc release (reusing the existing find_nvcc_path() helper) and keep only the architectures that toolkit supports:

nvcc CUDA	default archs
≥ 12.0	80;86;89;90 (unchanged)
11.8–11.x	80;86;89
11.1–11.7	80;86
11.0	80

Behavior is preserved everywhere it matters:

• An explicitly set CPUINFER_CUDA_ARCHS still wins — untouched.
• On a modern toolkit (≥ 12.0) the default is byte-for-byte the old 80;86;89;90.
• If nvcc can't be probed, we fall back to the previous full list, so nothing regresses when detection is unavailable.

A one-line note is printed when the default is narrowed, pointing users at CPUINFER_CUDA_ARCHS if they want to override.

Fixes #2098.

🤖 Generated with Claude Code

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

[AmirF194]

The default derivation looks right (checked the sm_80/86/89/90 CUDA-version gates against NVIDIA's own release notes, and they match). One gap in the "explicitly set wins, untouched" claim though.

os.environ.get("CPUINFER_CUDA_ARCHS", "").strip() or _default_cuda_archs() can't tell "unset" from "set to an empty string". Before this PR, CPUINFER_CUDA_ARCHS="" was the documented way to suppress -DCMAKE_CUDA_ARCHITECTURES entirely (the comment above says "if users want to specify CUDA archs, they can set env CPUINFER_CUDA_ARCHS"), since os.environ.get(key, default) only substitutes when the key is absent. With this change, that same explicit empty value now falls through to _default_cuda_archs() and gets a computed arch list instead of no flag at all.

Confirmed with the two snippets isolated from main vs this branch, CPUINFER_CUDA_ARCHS="" in both:

main:  cmake_args: []
PR:    cmake_args: ['-DCMAKE_CUDA_ARCHITECTURES=80;86;89;90']

Distinguishing "unset" from "empty" ("CPUINFER_CUDA_ARCHS" in os.environ before the .strip()) would keep the explicit opt-out working. Not blocking, this is a narrow edge case, but worth a line since the PR body specifically claims that path is untouched.

[Anai-Guo]

Good catch — you're right, and the PR body's claim was wrong on that path.

os.environ.get("CPUINFER_CUDA_ARCHS", "") returns the empty string both when the key is absent and when it's explicitly set to "", so the or _default_cuda_archs() fallback swallowed the documented opt-out. Fixed in 0a8d469 by checking key presence before stripping:

if "CPUINFER_CUDA_ARCHS" in os.environ:
    archs_env = os.environ["CPUINFER_CUDA_ARCHS"].strip()
else:
    archs_env = _default_cuda_archs()

Re-ran your snippet comparison plus a couple more values:

CPUINFER_CUDA_ARCHS	main	this branch
unset	-DCMAKE_CUDA_ARCHITECTURES=80;86;89;90	nvcc-derived list
""	(no flag)	(no flag)
" "	(no flag)	(no flag)
"89"	...=89	...=89

So the only behavior change left is the unset case, which is what #2098 is about.

[greptile-apps]

When ver < (11, 0) all four conditions are false so archs stays empty, and selected falls back to full ("80;86;89;90"). This means a CUDA 10.x toolkit would still get the full arch list, causing the same nvcc rejection the PR aims to fix. In practice CUDA 10.x predates sm_80 itself (A100 launched with CUDA 11.0), so the build would fail for other reasons too, but the fallback could silently produce a misleading error rather than a clear "unsupported toolkit" message. Adding an explicit guard or a warning for ver < (11, 0) would make the failure more actionable.

Prompt To Fix With AI

This is a comment left during a code review.
Path: kt-kernel/setup.py
Line: 703-712

Comment:
When `ver < (11, 0)` all four conditions are false so `archs` stays empty, and `selected` falls back to `full` (`"80;86;89;90"`). This means a CUDA 10.x toolkit would still get the full arch list, causing the same `nvcc` rejection the PR aims to fix. In practice CUDA 10.x predates `sm_80` itself (A100 launched with CUDA 11.0), so the build would fail for other reasons too, but the fallback could silently produce a misleading error rather than a clear "unsupported toolkit" message. Adding an explicit guard or a warning for `ver < (11, 0)` would make the failure more actionable.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[greptile-apps]

Greptile Summary

This PR fixes a build failure on older CUDA toolkits (e.g. CUDA 11.5) by replacing the hard-coded 80;86;89;90 default arch list with one derived from the detected nvcc version at build time. The existing CPUINFER_CUDA_ARCHS override path is preserved unchanged.

• A new inner function _default_cuda_archs() runs nvcc --version, parses the release string, and returns only the architectures that toolkit supports, falling back to the full list if detection fails.
• The version gate for sm_90 uses (12, 0), but CUDA 11.8 is the first release with Hopper (sm_90) support; sm_90a (extended Hopper) is what CUDA 12.0 added. This means CUDA 11.8/11.x builds targeting H100 silently omit Hopper-optimized code.

Confidence Score: 4/5

The core detection logic is sound, but the sm_90 version gate is wrong: it excludes Hopper kernels on CUDA 11.8/11.x, which is one of the toolkit generations this fix is specifically intended to help.

The sm_90 guard uses CUDA 12.0 as the cutoff, but NVIDIA introduced sm_90 support in CUDA 11.8. A user building on CUDA 11.8 with an H100 will get a wheel that silently omits Hopper-optimized code, since archs will contain 80;86;89 rather than 80;86;89;90.

Files Needing Attention: kt-kernel/setup.py — the version comparison for sm_90 in _default_cuda_archs

Important Files Changed

Filename	Overview
kt-kernel/setup.py	Adds nvcc version detection to derive safe default CUDA arch list; the sm_90 version gate is wrong (guard uses 12.0, but sm_90 support was added in CUDA 11.8), causing H100 kernels to be silently dropped on CUDA 11.8/11.x toolkits.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[CPUINFER_USE_CUDA=ON] --> B{CPUINFER_CUDA_ARCHS set?}
    B -- Yes --> C[Use env value as-is]
    B -- No --> D[_default_cuda_archs nvcc_path]
    D --> E{nvcc resolvable?}
    E -- No --> F[fallback: 80;86;89;90]
    E -- Yes --> G[run nvcc --version]
    G --> H{parse release X.Y}
    H -- parse fails --> F
    H -- success --> I{ver >= 11.0?}
    I -- No --> F
    I -- Yes --> J[archs = 80]
    J --> K{ver >= 11.1?}
    K -- Yes --> L[add 86]
    K -- No --> M[return joined archs]
    L --> N{ver >= 11.8?}
    N -- Yes --> O[add 89 and 90]
    N -- No --> M
    O --> M
    C --> R{archs_env non-empty AND CMAKE_CUDA_ARCHITECTURES not in args?}
    M --> R
    F --> R
    R -- Yes --> S[append -DCMAKE_CUDA_ARCHITECTURES]
    R -- No --> T[skip]

Loading

_{Reviews (2): Last reviewed commit: "refactor(kt-kernel): reuse resolved nvcc..." | Re-trigger Greptile}

[Anai-Guo]

Thanks @greptile — good catch on Issue 1. Fixed in f827bd3: _default_cuda_archs() now takes the nvcc_path already resolved for -DCMAKE_CUDA_COMPILER and only falls back to a fresh find_nvcc_path() when the compiler was user-specified (so we never resolved a path ourselves). This drops the duplicate filesystem probes and guarantees the arch-detection and the compiler flag use the same binary.

🤖 Generated with Claude Code

[greptile-apps]

The fix in f827bd3 looks correct. The flow is clean:

1. nvcc_path = None initialized before the compiler detection block
2. find_nvcc_path() runs once and assigns to nvcc_path (only when not user-specified)
3. _default_cuda_archs(nvcc_path) receives the already-resolved path — no second probe in the common case
4. The fallback nvcc = nvcc or find_nvcc_path() inside the helper only triggers when nvcc_path is None, i.e. when the user provided their own CMAKE_CUDA_COMPILER and we never resolved a path

This guarantees both -DCMAKE_CUDA_COMPILER and -DCMAKE_CUDA_ARCHITECTURES always reference the same binary. The implementation is solid — thanks for the follow-up.

Tip: You can customize Greptile's behavior for this repo with .greptile/rules.md and .greptile/config.json.

[Anai-Guo]

Friendly ping @maintainers — CI here is gated on the run-ci label (check-changes fails with "This pull request does not have the 'run-ci' label"), so the red checks aren't from the change itself and I can't unblock them from my side. Could someone add the label so the kt-kernel build actually runs?

The review feedback has been addressed in f827bd3 and the change is scoped to _default_cuda_archs() picking the default CMAKE_CUDA_ARCHITECTURES from the detected nvcc version, so older toolkits (CUDA 11.5) stop failing on unsupported archs.