Compiled ik_llama does not load models properly
cd projects
git clone https://github.com/ikawrakow/ik_llama.cpp.git
cd ik_llama.cpp
git fetch origin
git remote add ubergarm https://github.com/ubergarm/ik_llama.cpp
git fetch ubergarm
git checkout ug/hunyuan-moe-2
git checkout -b merge-stuff-here
git merge ikawrakow/ik/iq3_ks_v2
build for CUDA
cmake -B build -DCMAKE_BUILD_TYPE=Release -DGGML_CUDA=ON -DGGML_VULKAN=OFF -DGGML_RPC=OFF -DGGML_BLAS=OFF -DGGML_CUDA_F16=ON -DGGML_SCHED_MAX_COPIES=1
cmake --build build --config Release -j 20
Run the git merge ikawrakow/ik/iq3_ks_v2 command,Show hint: merge: ikawrakow/ik/iq3_ks_v2 - not something we can merg
Yes there was a mistake in the exact command, but it is just git merge stuff so take moment and think. Also this already changed in the past few hours, I'll update README with accurate instructions now that I have sleep! Thanks!