fix(hipblas): symlink amdgpu.ids so ROCm backends find the ASIC ID table#10627
Conversation
ROCm's bundled libdrm_amdgpu looks up the GPU ASIC ID table at a hardcoded fallback path, /opt/amdgpu/share/libdrm/amdgpu.ids, which is only populated by AMD's full amdgpu-install (graphics/DKMS) stack. The hipblas image is compute-only and doesn't have it, so every model load logs "No such file or directory" and the GPU can't be identified. Symlink it to the equivalent file already shipped by Ubuntu's libdrm-amdgpu1 package. Fixes #10624 Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
Verified against the real rocm/dev-ubuntu-24.04:7.2.1 image with hipblas-dev/hipblaslt-dev/rocblas-dev installed: /usr/share/libdrm/amdgpu.ids is owned by libdrm-common, not libdrm-amdgpu1 as the comment said. Signed-off-by: Ettore Di Giacinto <mudler@localai.io>
|
hi @mudler The Python backends (such as vLLM and sglang) build hipblas variants and will face the same GPU identification warning or failure if the amdgpu.ids symlink is missing there. |
backends Dockerfiles are used to create containers from-scratch. these images are not executed as-is, so moving the files there has no effect. IF |
Summary
libdrm_amdgpulooks up the GPU ASIC ID table at a hardcoded fallback path,/opt/amdgpu/share/libdrm/amdgpu.ids, which only exists if AMD's fullamdgpu-install(graphics/DKMS) stack is installedhipblasimage (rocm/dev-ubuntu-24.04base) is compute-only, so that path never exists, and every model load on a hipblas backend logs/opt/amdgpu/share/libdrm/amdgpu.ids: No such file or directorylibdrm-commonpackage ships the equivalent data at/usr/share/libdrm/amdgpu.ids; symlink it into place forBUILD_TYPE=hipblas, guarded so it's a no-op if either path is already in an unexpected stateBackends run as plain child processes of the main
local-aiprocess and share its root filesystem (confirmed by tracingpkg/oci/image.goextraction +pkg/model/process.go/core/cli/workerexec), so a single symlink at the top-levelDockerfile(requirements-driversstage) covers all hipblas backends (diffusers, llama.cpp, etc.) running underlocalai/localai:*-gpu-hipblas.Fixes #10624
Test plan
rocm/dev-ubuntu-24.04:7.2.1directly and ranapt-get install hipblas-dev hipblaslt-dev rocblas-dev(the exact packages this Dockerfile installs forBUILD_TYPE=hipblas), then confirmed:/opt/amdgpu/share/libdrm/does not exist (reproduces the reported bug) and/usr/share/libdrm/amdgpu.idsexists, owned bylibdrm-common(dpkg -S amdgpu.ids)RUNshell logic against a realdocker build(mocked/usr/share/libdrm/amdgpu.idspresent) — symlink is created correctly and resolves*-gpu-rocm-hipblas-llama-cppbackend image to confirmlibdrm_amdgpuis not statically linked/bundled into the backend's ownlib/, so its lookup relies on the outer image's filesystem, matching where this fix liveslocalai/localai:*-gpu-hipblasimage build + on-hardware ROCm verification (needs AMD GPU, not available in this environment)