Hybrid EP:Support configurable IB Traffic Class via HYBRID_EP_IB_TC env var#611
Open
kzlxd wants to merge 1 commit into
Open
Hybrid EP:Support configurable IB Traffic Class via HYBRID_EP_IB_TC env var#611kzlxd wants to merge 1 commit into
kzlxd wants to merge 1 commit into
Conversation
shsjshentao
pushed a commit
to shsjshentao/DeepEP
that referenced
this pull request
May 15, 2026
Cherry-pick deepseek-ai#611 (kzlxd:hybrid_ep_add_env_for_tc, still open upstream) to address two-node dispatch throughput anomalies observed when TC=0 is hardcoded on B300. - Drop constexpr DEF_IB_TC, add get_ib_tc_value() reading HYBRID_EP_IB_TC env var with range validation [0, 255] and default 0 fallback (warns on invalid input to stderr). - Use get_ib_tc_value() at the doca_verbs_ah_attr_set_traffic_class call site in setup_qp_attr_for_modify. - Swap our prior <torch/torch.h> include for <torch/extension.h> (PR's choice) and add <climits>, so this file is byte-identical to PR head and pulls cleanly when the PR merges upstream. Default behavior unchanged (TC=0). NIXL path unaffected — only the DOCA backend compiles internode_doca.cu. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds support for configuring the IB Traffic Class (TC) used in RDMA QP setup via an environment variable, replacing the previously hardcoded value.
HYBRID_EP_IB_TCfor runtime configurationDEF_IB_TC = 0get_ib_tc_value()with range validation (0–255)This change was motivated by observed performance anomalies when using
TC=0(e.g., ~46 GB/s in 2-node dispatch scenarios), which revealed that the TC value was hardcoded.Changes
IB TC Configuration
csrc/hybrid_ep/buffer/internode_doca.cuhDEF_IB_TCget_ib_tc_value()csrc/hybrid_ep/buffer/internode_doca.cuImplement
get_ib_tc_value():HYBRID_EP_IB_TCfrom environment[0, 255]0if invalid (with warning)Replace all usages of
DEF_IB_TCwithget_ib_tc_value()Add missing includes:
#include <torch/extension.h>
#include
Usage
[0, 255]will:0Performance Impact
TC=0): ~46 GB/s (2-node dispatch)TC=162): significantly improved throughputNote: exact gains depend on network configuration and topology.
Notes
TC=0)