Support dense THD stats graphs on SM8x - #5
Draft
sudhakarsingh27 wants to merge 2 commits into
Draft
Conversation
cuDNN accepts THD on SM8x from 9.18.1 but requires dense Stats/LSE and max-sequence graph dimensions there. Use one representation predicate for graph construction, auxiliary allocation, and cache-key normalization so support probing and execution cannot diverge. Broaden the shared Python THD layout filter to mixed Q/KV layouts and add focused predicate, padding-filter, and SM8x support-boundary coverage. Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
Keep the cross-fork follow-up limited to the requested production-code changes. Signed-off-by: Sudhakar Singh <sudhakars@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
B x H x S_q x 1Stats/Max tensors and max-sequence graph dimensionsWhy
cuDNN 9.18.1 enables THD inputs on SM8x, but packed/ragged LSE remains unsupported for backward propagation there. The PR 2964 graph-as-the-source-of-truth approach is reliable only when the probed graph uses the runtime-supported representation. Its cache key must normalize the same dimensions that graph construction uses.
This intentionally does not add a separate hard-coded SM8x/cuDNN fused-support table: the forward and backward graph probes remain authoritative.
Validation
git diff --checkpassed