Skip to content

Conversation

@KiruyaMomochi
Copy link

@KiruyaMomochi KiruyaMomochi commented Nov 13, 2025

The implementation might support Kimi-K2-Instruct too, but I don't have enough disk space to test now :(

Almost silly copy-paste from DeepSeek V3.1 #15533, modified according to https://github.com/MoonshotAI/Kimi-K2/blob/main/docs/tool_call_guidance.md: matching function id instead of plain function name.

Considerations:

  1. The official template does not contain any <think> tag at the end, so thinking_forced_open is false. Should we test it by modify the template manually?
  2. Did not add template update instruction to models/templates/README.md for now, because their template has tojson(separators=(',', ':')). Although the value of separators is the same as default value, but we must remove it to make the template work for minja.
  3. DeepSeek V3.1 might be possible to generate <|tool▁calls▁begin|>tool... and ignoring <|tool▁call▁begin|>, but I have not observed such behavior in Kimi-K2-Thinking and always get <|tool_calls_section_begin|><|tool_call_begin|>, therefore I'm removing the ? in the function regex.
    static const common_regex function_regex("(?:<|tool▁call▁begin|>)?([^\\n<]+)(?:<|tool▁sep|>)");

    Actually, I always get an extra <|tool_calls_section_end|> when keeping ?, but I have not been able to fix it, so finally removed the ?.
  4. Have not tested lower quantized variants, maybe they could have different behavior which need to adapt the current parser?

For maintainers: I may have a busy weekend so fell free to edit directly if I'm not able to reply in time.

Closes #17155.

@calvin2021y
Copy link

#16932

@KiruyaMomochi
Copy link
Author

Thanks! Will try #16932.

@KiruyaMomochi
Copy link
Author

Finally get some to test and it worked well!

@KiruyaMomochi
Copy link
Author

KiruyaMomochi commented Nov 15, 2025

Close in favour of #16932.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Request: Kimi-K2-Thinking reasoning and tool calling support

2 participants