Description
Git commit
Termux again. ChatGPT wrote most of below:
Title: Vulkan shaders build fails due to missing vulkan-shaders directory during ExternalProject_Add configure step
Description:
When building with -DGGML_VULKAN=ON, the build fails during the vulkan-shaders-gen ExternalProject configuration step with this error:
[ 3%] Built target ggml-base
[ 4%] Creating directories for 'vulkan-shaders-gen'
[ 4%] No download step for 'vulkan-shaders-gen'
[ 5%] No update step for 'vulkan-shaders-gen'
[ 5%] No patch step for 'vulkan-shaders-gen'
[ 5%] Performing configure step for 'vulkan-shaders-gen'
-- The C compiler identification is Clang 20.1.5
-- The CXX compiler identification is Clang 20.1.5
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /data/data/com.termux/files/usr/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /data/data/com.termux/files/usr/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:3 (add_subdirectory):
add_subdirectory given source "vulkan-shaders" which is not an existing
directory.
Workaround:
Manual compilation of vulkan-shaders-gen is required:
-
Manually build
vulkan-shaders-geninside
/data/data/com.termux/files/home/downloads/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/ -
Copy the executable to the expected binary directory before resuming the main build:
cp /data/data/com.termux/files/home/downloads/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/bin/vulkan-shaders-gen bin/Likely culprit:
The problem appears to originate from the CMake setup in the Vulkan backend, specifically either:
ggml/src/CMakeLists.txtaround theExternalProject_Add(vulkan-shaders-gen ...)- or
ggml/src/ggml-vulkan/CMakeLists.txtwhich handles shader building and thevulkan-shaderssubdirectory references.
The build system tries to add_subdirectory("vulkan-shaders") in a path where that folder is missing, causing the configure step failure.
Request:
Please review the Vulkan shader CMake setup to ensure that:
-
The
vulkan-shaderssource directory exists or is properly downloaded/copied before the configure step. -
The
ExternalProject_Addcommands correctly reference existing paths, avoiding the manual compile-and-copy workaround.
My take: it compiles with the bells and whistles on Termux: cmake -S . -B build -DLLAMA_CURL=ON -DGGML_VULKAN=ON -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=OpenBLAS -DGGML_CUDA=OFF -DGGML_ARCH_FLAGS="-march=armv8.2-a+dotprod+fp16" , but these resulting executables sometimes need hunting for by hand, due to some weird:
~/.../llama.cpp/build $ ls /data/data/com.termux/files/home/downloads/llama.cpp/build/bin/vulkan-shaders-gen
lsd: /data/data/com.termux/files/home/downloads/llama.cpp/build/bin/vulkan-shaders-gen: No such file or directory (os error 2).
~/.../llama.cpp/build $ cp /data/data/com.termux/files/home/downloads/llama.cpp/ggml/src/ggml-vulkan/vulkan-shaders/bin/vulkan-shaders-gen /data/data/com.termux/files/home/downloads/llama.cpp/build/bin/vulkan-shaders-gen
~/.../llama.cpp/build $ ls /data/data/com.termux/files/home/downloads/llama.cpp/build/bin/vulkan-shaders-gen
/data/data/com.termux/files/home/downloads/llama.cpp/build/bin/vulkan-shaders-gen
~/.../llama.cpp/build $ make install
[ 3%] Built target ggml-base
[ 6%] Built target vulkan-shaders-gen-build
[ 6%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "RelWithDebInfo"
CMake Error at cmake_install.cmake:52 (file):
file INSTALL cannot find
"/data/data/com.termux/files/home/downloads/llama.cpp/build/bin/vulkan-shaders-gen":
No such file or directory.
make[2]: *** [ggml/src/ggml-vulkan/CMakeFiles/vulkan-shaders-gen-install.dir/build.make:73: ggml/src/ggml-vulkan/vulkan-shaders-gen-prefix/src/vulkan-shaders-gen-stamp/vulkan-shaders-gen-install] Error 1
make[1]: *** [CMakeFiles/Makefile2:2390: ggml/src/ggml-vulkan/CMakeFiles/vulkan-shaders-gen-install.dir/all] Error 2
make: *** [Makefile:146: all] Error 2
~/.../llama.cpp/build $ make install
[ 3%] Built target ggml-base
[ 6%] Built target vulkan-shaders-gen-build
[ 6%] Performing install step for 'vulkan-shaders-gen'
-- Install configuration: "RelWithDebInfo"
CMake Error at cmake_install.cmake:52 (file):
file INSTALL cannot find
"/data/data/com.termux/files/home/downloads/llama.cpp/build/bin/vulkan-shaders-gen":
No such file or directory.
Operating systems
Other? (Please let us know in description)
GGML backends
Vulkan
Problem description & steps to reproduce
See above
First Bad Commit
No response
Compile command
cmake -S . -B build -DLLAMA_CURL=ON -DGGML_VULKAN=ONRelevant log output
See above