Skip to content

docs: add README build mode guidance - #2120

Open
zcxGGmu wants to merge 2 commits into
kvcache-ai:mainfrom
zcxGGmu:docs/build-mode-readme
Open

docs: add README build mode guidance#2120
zcxGGmu wants to merge 2 commits into
kvcache-ai:mainfrom
zcxGGmu:docs/build-mode-readme

Conversation

@zcxGGmu

@zcxGGmu zcxGGmu commented Aug 3, 2026

Copy link
Copy Markdown

Summary

  • Add README build-mode guidance for the inference quick start.
  • Show the recommended prebuilt install, a CPU-only / portable AVX2 source build, and a CUDA source build path.
  • Clarify that pure CPU experiments should use KT-Kernel CPU backends directly, while balance_serve remains a CPU-GPU heterogeneous serving path.

Refs #1187

Test Plan

  • python README guard check for the new build-mode text
  • ./install.sh --help output contains --manual
  • kt-kernel/install.sh --help output contains CPUINFER_CPU_INSTRUCT and CPUINFER_ENABLE_AMX
  • git diff --check HEAD~1..HEAD

@greptile-apps

greptile-apps Bot commented Aug 3, 2026

Copy link
Copy Markdown

Greptile Summary

This PR adds a new "Installation / build modes" section to the root README, documenting three build paths: a prebuilt pip install, a CPU-only AVX2 source build, and a CUDA source build for heterogeneous serving. It also clarifies the scope of balance_serve (CPU-GPU path, not standalone CPU).

  • CPU-only source build correctly goes into kt-kernel/ and calls ./install.sh all --manual, which runs install_dependencies before the build step — the previously flagged build --manual issue has been resolved.
  • CUDA source build uses the root ./install.sh all, which calls install_deps (system dependencies via kt-kernel/install.sh deps) before delegating the build to kt-kernel/install.sh build, so system prerequisites are correctly installed on a fresh environment.
  • Prebuilt path (pip install kt-kernel sglang-kt) is a straightforward one-liner that needs no further qualification.

Confidence Score: 5/5

Documentation-only change; all three build paths have been verified against the actual install.sh scripts and are correct.

The CPU-only build correctly uses all --manual (addressing the previously flagged issue), and the CUDA build's root ./install.sh all properly calls install_deps before delegating the build, so system prerequisites are installed on a fresh clone in both paths.

Files Needing Attention: No files require special attention.

Important Files Changed

Filename Overview
README.md Adds an 'Installation / build modes' section with three build paths (prebuilt, CPU-only source, CUDA source); commands are correct against the actual install.sh scripts

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[User wants to install ktransformers] --> B{Build path?}
    B --> C[Prebuilt]
    B --> D[CPU-only source]
    B --> E[CUDA source]

    C --> C1["pip install kt-kernel sglang-kt"]

    D --> D1["git clone --recursive"]
    D1 --> D2["cd ktransformers/kt-kernel"]
    D2 --> D3["CPUINFER_USE_CUDA=0 CPUINFER_CPU_INSTRUCT=AVX2\n./install.sh all --manual"]
    D3 --> D4["install_dependencies (libhwloc-dev, pkg-config)"]
    D4 --> D5["build_step --manual (AVX2, no CUDA)"]

    E --> E1["cd /path/to/ktransformers"]
    E1 --> E2["CPUINFER_USE_CUDA=1 ./install.sh all"]
    E2 --> E3["init_submodules"]
    E3 --> E4["install_deps - kt-kernel/install.sh deps"]
    E4 --> E5["install_sglang"]
    E5 --> E6["install_kt_kernel - kt-kernel/install.sh build (CUDA)"]
Loading

Reviews (2): Last reviewed commit: "[docs]: use all for fresh CPU build mode" | Re-trigger Greptile

Comment thread README.md Outdated
Comment thread README.md Outdated
@zcxGGmu

zcxGGmu commented Aug 4, 2026

Copy link
Copy Markdown
Author

Thanks for the review — I pushed 3c17240 addressing the fresh-clone build command feedback.

Changes made:

  • Updated the CPU-only source build example to use ./install.sh all --manual so dependency installation runs before the build step.

Validation:

  • grep -nF 'CPUINFER_USE_CUDA=0 CPUINFER_CPU_INSTRUCT=AVX2 CPUINFER_ENABLE_AMX=OFF ./install.sh all --manual' README.md — passed.
  • git diff --check — passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants