50-whisper.sh build error

[Soloturkh]

Whisper.pc is found, version is OK, libs/cflags are correct. But FFmpeg’s require_pkg_config call (inside configure) performs a link-time test after the pkg-config check — it tries to link a small test program against the whisper libraries. The linker reports that symbols inside libggml-vulkan.a such as matmul_id_subgroup_* are not defined → link failure → configure incorrectly reports “not found”.

So the real bug is:

The Vulkan backend in whisper.cpp commit 8443cf05e (v1.8.4) has an incomplete shader generation pipeline.

The vulkan-shaders-gen tool runs successfully (as seen in the build log: Completed 'vulkan-shaders-gen'), but it does not generate the required matmul_id_subgroup_* shader variants or is not aligned with the current GGML source.

As a result:

The library contains references to shader data symbols
But the actual definitions of those symbols are missing
→ This leads to a linker failure.

/opt/ct-ng/x86_64-ffbuild-linux-gnu/bin/ld: ggml-vulkan.cpp:(.text+0xa1941):
undefined reference to matmul_id_subgroup_iq3_s_f32_len' /opt/ct-ng/x86_64-ffbuild-linux-gnu/bin/ld: ggml-vulkan.cpp:(.text+0xa199c): undefined reference to matmul_id_subgroup_iq3_s_f32_data'
... matmul_id_subgroup_f16_f32_aligned_f16acc_len ...
... matmul_id_subgroup_f16_f32_f16acc_len ...

[Soloturkh]

#!/bin/bash

SCRIPT_REPO="https://github.com/ggml-org/whisper.cpp.git"
SCRIPT_COMMIT="8443cf05e3fa8ce1b32348e1bcbcf8fc31f7f3ae"

ffbuild_depends() {
    echo base
    echo opencl
}

ffbuild_enabled() {
    [[ $TARGET != *32 ]] || return -1
    (( $(ffbuild_ffver) >= 800 )) || return -1
    return 0
}

ffbuild_dockerbuild() {
    mkdir build && cd build

    cmake -GNinja -DCMAKE_TOOLCHAIN_FILE="$FFBUILD_CMAKE_TOOLCHAIN" -DCMAKE_INSTALL_PREFIX="$FFBUILD_PREFIX" -DCMAKE_BUILD_TYPE=Release \
        -DBUILD_SHARED_LIBS=OFF -DWHISPER_BUILD_TESTS=OFF -DWHISPER_BUILD_EXAMPLES=OFF -DWHISPER_BUILD_SERVER=OFF -DWHISPER_USE_SYSTEM_GGML=OFF \
        -DGGML_CCACHE=OFF -DGGML_OPENCL=ON -DGGML_VULKAN=OFF \
        -DGGML_NATIVE=OFF -DGGML_SSE42=ON -DGGML_AVX=ON -DGGML_F16C=ON -DGGML_AVX2=ON -DGGML_BMI2=ON -DGGML_FMA=ON ..

    ninja -j$(nproc)
    DESTDIR="$FFBUILD_DESTDIR" ninja install

    # For some reason, these lack the lib prefix on Windows
    shopt -s nullglob
    for libfile in "$FFBUILD_DESTPREFIX"/lib/ggml*.a; do
        mv "${libfile}" "$(dirname "${libfile}")/lib$(basename "${libfile}")"
    done

    # Linking order is all wrong
    sed -i -e 's/^\(Libs:\).*$/\1 -L${libdir} -lwhisper/' "$FFBUILD_DESTPREFIX"/lib/pkgconfig/whisper.pc
    echo "Libs.private: -lggml -lggml-base -lggml-cpu -lggml-opencl -lstdc++" >> "$FFBUILD_DESTPREFIX"/lib/pkgconfig/whisper.pc
    echo "Requires: OpenCL" >> "$FFBUILD_DESTPREFIX"/lib/pkgconfig/whisper.pc
}

ffbuild_configure() {
    echo --enable-whisper
}

ffbuild_unconfigure() {
    (( $(ffbuild_ffver) >= 800 )) || return 0
    echo --disable-whisper
}

[BtbN]

Their build system is just busted and for some reason non-deterministically fails.
Just re-build it and it works. I had this failure happen on exactly one of all the images built on CI, so it's luckily pretty rare.

[Soloturkh]

It's obvious that they were 5 6 times I tried it from zero and it caused constant interruption.

[BtbN]

Well, I encountered it exactly once out of all the image builds that are on CI.
I just re-built that image, and it worked.
As you see, CI is green here currently.
Make sure you actually re-build the image, and not just use the already broken cache.