Skip to content

Commit b1897ef

Browse files
committed
Add mistral 7b example
1 parent 1498c00 commit b1897ef

File tree

5 files changed

+17
-3
lines changed

5 files changed

+17
-3
lines changed
Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Start-Process "http://127.0.0.1:8080"
2+
3+
../vendor/llama.cpp/build/bin/Release/server `
4+
--model "../vendor/llama.cpp/models/FashionGPT-70B-V1.1/model-quantized-q4_K_M.gguf" `
5+
--ctx-size 4096 `
6+
--threads 16 `
7+
--n-gpu-layers 10

examples/server_mistral_7b_v0.1.ps1

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Start-Process "http://127.0.0.1:8080"
2+
3+
../vendor/llama.cpp/build/bin/Release/server `
4+
--model "../vendor/llama.cpp/models/Mistral-7B-v0.1/model-quantized-q4_K_M.gguf" `
5+
--ctx-size 4096 `
6+
--threads 16 `
7+
--n-gpu-layers 35

examples/server_phind_codellama_34b_v2_32K.ps1

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@ Start-Process "http://127.0.0.1:8080"
88
#
99
../vendor/llama.cpp/build/bin/Release/server `
1010
--model "../vendor/llama.cpp/models/Phind-CodeLlama-34B-v2/model-quantized-q4_K_M.gguf" `
11-
--ctx-size 16384 `
11+
--ctx-size 32768 `
1212
--rope-freq-scale 0.125 `
1313
--rope-freq-base 80000 `
1414
--threads 16 `

examples/server_wizardcoder_15b_v1.0.ps1

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,5 +4,5 @@ Start-Process "http://127.0.0.1:8080"
44
../vendor/llama.cpp/build/bin/Release/server `
55
--model "../vendor/llama.cpp/models/WizardCoder-15B-V1.0/model-quantized-q4_k_M.gguf" `
66
--ctx-size 2048 `
7-
--threads 6 `
7+
--threads 16 `
88
--n-gpu-layers 0

vendor/llama.cpp

0 commit comments

Comments
 (0)