Kimi-K2-Thinking native tool calling format #17251
Closed
+584
−3
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The implementation might support Kimi-K2-Instruct too, but I don't have enough disk space to test now :(
Almost silly copy-paste from DeepSeek V3.1 #15533, modified according to https://github.com/MoonshotAI/Kimi-K2/blob/main/docs/tool_call_guidance.md: matching function id instead of plain function name.
Considerations:
<think>tag at the end, sothinking_forced_openis false. Should we test it by modify the template manually?tojson(separators=(',', ':')). Although the value ofseparatorsis the same as default value, but we must remove it to make the template work for minja.<|tool▁calls▁begin|>tool...and ignoring<|tool▁call▁begin|>, but I have not observed such behavior in Kimi-K2-Thinking and always get<|tool_calls_section_begin|><|tool_call_begin|>, therefore I'm removing the?in the function regex.llama.cpp/common/chat.cpp
Line 1751 in c4abcb2
Actually, I always get an extra
<|tool_calls_section_end|>when keeping?, but I have not been able to fix it, so finally removed the?.For maintainers: I may have a busy weekend so fell free to edit directly if I'm not able to reply in time.
Closes #17155.