Skip to content

Commit 1475728

Browse files
committed
Add -disableThinking option
1 parent f145775 commit 1475728

File tree

2 files changed

+14
-1
lines changed

2 files changed

+14
-1
lines changed

examples/server.ps1

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,12 @@ Param (
122122
[String]
123123
$kvCacheDataType="f16",
124124

125+
[Parameter(
126+
HelpMessage="Disables the thinking mode of the model."
127+
)]
128+
[switch]
129+
$disableThinking=$false,
130+
125131
[switch]
126132
$help,
127133

@@ -135,6 +141,7 @@ if ($help) {
135141
exit
136142
}
137143

144+
138145
# The -verbose option is a default PowerShell parameter.
139146
$verbose = $PSCmdlet.MyInvocation.BoundParameters["Verbose"].IsPresent -eq $true
140147

@@ -331,6 +338,12 @@ if ($enableFlashAttention) {
331338
$commandArguments += "--flash-attn"
332339
}
333340

341+
# @see https://github.com/ggml-org/llama.cpp/pull/13771#issuecomment-2909087077
342+
if ($disableThinking) {
343+
$commandArguments += "--jinja"
344+
$commandArguments += "--reasoning-budget 0"
345+
}
346+
334347
if ($verbose) {
335348
$commandArguments += "--verbose"
336349
}

vendor/llama.cpp

Submodule llama.cpp updated 626 files

0 commit comments

Comments
 (0)