Skip to content

Hybrid EP:Support configurable IB Traffic Class via HYBRID_EP_IB_TC env var#611

Open
kzlxd wants to merge 1 commit into
deepseek-ai:hybrid-epfrom
kzlxd:hybrid_ep_add_env_for_tc
Open

Hybrid EP:Support configurable IB Traffic Class via HYBRID_EP_IB_TC env var#611
kzlxd wants to merge 1 commit into
deepseek-ai:hybrid-epfrom
kzlxd:hybrid_ep_add_env_for_tc

Conversation

@kzlxd

@kzlxd kzlxd commented Apr 24, 2026

Copy link
Copy Markdown

Summary

This PR adds support for configuring the IB Traffic Class (TC) used in RDMA QP setup via an environment variable, replacing the previously hardcoded value.

  • Add HYBRID_EP_IB_TC for runtime configuration
  • Remove hardcoded DEF_IB_TC = 0
  • Introduce get_ib_tc_value() with range validation (0–255)
  • Fix DOCA compilation issue by adding missing headers

This change was motivated by observed performance anomalies when using TC=0 (e.g., ~46 GB/s in 2-node dispatch scenarios), which revealed that the TC value was hardcoded.


Changes

IB TC Configuration

csrc/hybrid_ep/buffer/internode_doca.cuh

  • Remove DEF_IB_TC
  • Declare get_ib_tc_value()

csrc/hybrid_ep/buffer/internode_doca.cu

  • Implement get_ib_tc_value():

    • Read HYBRID_EP_IB_TC from environment
    • Validate range [0, 255]
    • Fall back to 0 if invalid (with warning)
  • Replace all usages of DEF_IB_TC with get_ib_tc_value()

  • Add missing includes:

    #include <torch/extension.h>
    #include


Usage

# Optional: set IB Traffic Class (default = 0)
export HYBRID_EP_IB_TC=162

python -u tests/test_hybrid_ep.py --num-processes 8
  • Values outside [0, 255] will:
    • Trigger a warning
    • Fall back to 0

Performance Impact

  • Before (TC=0): ~46 GB/s (2-node dispatch)
  • After (e.g., TC=162): significantly improved throughput

Note: exact gains depend on network configuration and topology.


Notes

  • Aligns with NVSHMEM-style runtime configurability
  • Helps avoid performance issues on certain platforms (e.g., B300 with TC=0)
  • Backward compatible (default remains TC=0)

@kzlxd kzlxd changed the title Support configurable IB Traffic Class via HYBRID_EP_IB_TC env var Hybrid EP:Support configurable IB Traffic Class via HYBRID_EP_IB_TC env var Apr 24, 2026
shsjshentao pushed a commit to shsjshentao/DeepEP that referenced this pull request May 15, 2026
Cherry-pick deepseek-ai#611 (kzlxd:hybrid_ep_add_env_for_tc,
still open upstream) to address two-node dispatch throughput
anomalies observed when TC=0 is hardcoded on B300.

- Drop constexpr DEF_IB_TC, add get_ib_tc_value() reading
  HYBRID_EP_IB_TC env var with range validation [0, 255] and
  default 0 fallback (warns on invalid input to stderr).
- Use get_ib_tc_value() at the doca_verbs_ah_attr_set_traffic_class
  call site in setup_qp_attr_for_modify.
- Swap our prior <torch/torch.h> include for <torch/extension.h>
  (PR's choice) and add <climits>, so this file is byte-identical
  to PR head and pulls cleanly when the PR merges upstream.

Default behavior unchanged (TC=0). NIXL path unaffected — only
the DOCA backend compiles internode_doca.cu.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant