Add initial version

Author: countzero

CHANGELOG.md

@@ -0,0 +1,11 @@
+# Changelog
+All notable changes to this project will be documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
+and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [1.0.0] - 2023-06-28
+
+### Added
+- OpenBLAS workaround for Windows
+- Rebuild script

README.md

@@ -1,6 +1,16 @@
 # Windows llama.cpp
 
-Some PowerShell automation to rebuild [llama.cpp](https://github.com/ggerganov/llama.cpp) for a Windows environment.
+A PowerShell automation to rebuild [llama.cpp](https://github.com/ggerganov/llama.cpp) for a Windows environment. It automates the following steps:
+
+1. Fetching and extracting a specific release of [OpenBLAS](https://github.com/xianyi/OpenBLAS/releases)
+2. Fetching the latest version of [llama.cpp](https://github.com/ggerganov/llama.cpp)
+3. Fixing OpenBLAS binding in the `CMakeLists.txt`
+4. Rebuilding the binaries with CMake
+5. Updating the Python dependencies
+
+## BLAS support
+
+This script currently supports `OpenBLAS` for CPU BLAS acceleration and `cuBLAS` for NVIDIA GPU BLAS acceleration.
 
 ## Installation
 
@@ -11,7 +21,7 @@ Download and install the latest versions:
 * [CMake](https://cmake.org/download/)
 * [Cuda](https://developer.nvidia.com/cuda-downloads)
 * [Git Large File Storage](https://git-lfs.com)
-* [Git](https://git-scm.com/download^^)
+* [Git](https://git-scm.com/download)
 * [Miniconda](https://conda.io/projects/conda/en/stable/user-guide/install)
 * [Visual Studio 2022 - Community](https://visualstudio.microsoft.com/downloads/)
 
@@ -60,10 +70,10 @@ conda init
 
 ### 6. Execute the build script
 
-To build llama.cpp binaries for a Windows environment with CUDA support execute the script:
+To build llama.cpp binaries for a Windows environment with CUDA BLAS acceleration execute the script:
 
 ```PowerShell
-./rebuild_llama.cpp.ps1
+./rebuild_llama.cpp.ps1 -blasAccelerator "cuBLAS"
 ```
 
 ### 7. Download a large language model
@@ -95,12 +105,10 @@ You can now chat with the model:
 
 ### Rebuild llama.cpp
 
-Every time there is a new release of [llama.cpp](https://github.com/ggerganov/llama.cpp) you can simply execute the script to automatically:
+Every time there is a new release of [llama.cpp](https://github.com/ggerganov/llama.cpp) you can simply execute the script to automatically rebuild everything:
 
-1. fetch the latest changes
-2. rebuild the binaries
-3. update the Python dependencies
-
-```PowerShell
-./rebuild_llama.cpp.ps1
-```
+| Command                                               | Description                |
+| ----------------------------------------------------- | -------------------------- |
+| `./rebuild_llama.cpp.ps1`                             | Without BLAS acceleration  |
+| `./rebuild_llama.cpp.ps1 -blasAccelerator "OpenBLAS"` | With CPU BLAS acceleration |
+| `./rebuild_llama.cpp.ps1 -blasAccelerator "cuBLAS"`   | With GPU BLAS acceleration |

rebuild_llama.cpp.ps1

@@ -1,3 +1,28 @@
+#Requires -Version 5.0
+
+<#
+.SYNOPSIS
+Automatically rebuild llama.cpp for a Windows environment.
+
+.DESCRIPTION
+This script automatically rebuilds llama.cpp for a Windows environment.
+
+.PARAMETER blasAccelerator
+Specifies the BLAS accelerator, supported values are: "OpenBLAS", "cuBLAS"
+
+.EXAMPLE
+.\rebuild_llama.ps1 -blasAccelerator "OpenBLAS"
+
+.EXAMPLE
+.\rebuild_llama.ps1 -blasAccelerator "cuBLAS"
+#>
+
+Param (
+    [ValidateSet("OpenBLAS", "cuBLAS")]
+    [String]
+    $blasAccelerator
+)
+
 $openBLASVersion = "0.3.23"
 
 if (-not(Test-Path -Path "./vendor/OpenBLAS/OpenBLAS-${openBLASVersion}-x64.zip")) {
@@ -48,11 +73,20 @@ New-Item -Path "./vendor/llama.cpp" -Name "build" -ItemType "directory"
 
 Set-Location -Path "./vendor/llama.cpp/build"
 
-cmake `
-    -DLLAMA_CUBLAS=OFF `
-    -DLLAMA_BLAS=ON `
-    -DLLAMA_BLAS_VENDOR=OpenBLAS `
-    ..
+switch ($blasAccelerator) {
+
+    "OpenBLAS" {
+        cmake -DLLAMA_BLAS=ON -DLLAMA_BLAS_VENDOR=OpenBLAS ..
+    }
+
+    "cuBLAS" {
+        cmake -DLLAMA_CUBLAS=ON ..
+    }
+
+    default {
+        cmake ..
+    }
+}
 
 cmake --build . --config Release