Getting error in build step

[sandipto-sanyal]

Hi Team,
I am facing error while performing the build step using
python setup_env.py -md models/BitNet-b1.58-2B-4T -q i2_s. Attached the log
compile.log

Attached my version checks

[cmlpolymath]

I faced similar issues during the setup_env.py step, and the CMake compilation as far as I'm concerned is a bit of a black box. Here's what you can do in order of easiest implementations:

1. The CMakeLists.txt in llama.cpp explicitly disables exceptions via -fno-exceptions. Add -DLLAMA_FORCE_LLVM=OFF (this is likely found in the CMakeLists.txt). If you take a look at the setup_env.py function, you gain a lot of insight into how it is creating paths and how cmake/clang are creating binaries.

Still, this is a known Windows/LLVM compatibility issue with llama.cpp. So if you're comfortable with Docker:

2. Try my docker image build (Future-proof Dockerization #227). Once the image is built, you can execute a run_inference.py command and you're up and running with the model.