ROCM Llama CPP backend missing file

[ALameLlama]

LocalAI version:
localai/localai:master-gpu-hipblas
sha256:4f477863d301e26317c1ffb99a3285d2acae841b7c87a4eb26578eaa518a6b84

Environment, CPU architecture, OS, and Version:
I'm doing this on unraid 7.3, using portainer.
AMD Ryzen 9 5900X
Radeon AI PRO R9700

Describe the bug
While from what I can tell everything works I think it's failing to find this file and using a fallback instead?

7:39:10 pm.745stderrrocblaslt error: Cannot read "TensileLibrary_lazy_gfx1201.dat": No such file or directory
7:39:10 pm.745stderrrocblaslt error: Could not load "TensileLibrary_lazy_gfx1201.dat"

To Reproduce
I just try and start any model on my node with AMD

Expected behavior
not to see an error

Logs
node-backend-logs-ornith-1.0-35b#0-2026-07-03.json

I've attached a log exported for the node

[localai-bot]

Thanks for the detailed report and the logs - they pinned it down exactly.

Root cause: The ROCm backend bundles its own copy of libhipblaslt.so under lib/, and it copied rocBLAS's kernel data (lib/rocblas/library/, wired up via ROCBLAS_TENSILE_LIBPATH) - but the parallel hipBLASLt data dir was never packaged and no HIPBLASLT_TENSILE_LIBPATH was set. So the bundled libhipblaslt.so looked for its per-arch kernel data relative to itself, found nothing, and silently fell back to slow generic kernels:

rocblaslt error: Cannot read "TensileLibrary_lazy_gfx1201.dat": No such file or directory

Your log confirmed the asymmetry - ROCBLAS_TENSILE_LIBPATH=/backends/rocm-llama-cpp/lib/rocblas/library is set and working, while hipBLASLt had no bundled data. This affected every gfx arch, not just gfx1201.

Fix (#10672):

• package-gpu-libs.sh now also bundles the hipblaslt/library/ Tensile data (mirroring the rocBLAS handling).
• llama-cpp/turboquant run.sh now export HIPBLASLT_TENSILE_LIBPATH pointing at the bundled dir.

Verified against the exact CI base image (rocm/dev-ubuntu-24.04:7.2.1): the fixed packager now produces lib/hipblaslt/library/TensileLibrary_lazy_gfx1201.dat (your R9700 is gfx1201, and ROCm 7.x does ship gfx1201 tensile data), which is exactly the file hipBLASLt was failing to open. The env var name/semantics were confirmed against hipBLASLt's tensile_host.cpp.

Once the PR is merged and a new master-gpu-hipblas image is published, the error should be gone (and you'll get the tuned kernels instead of the fallback). If you're able to test the resulting image on your R9700, that final on-hardware confirmation would be much appreciated - I don't have a gfx1201 card to run it on directly.

PR: #10672