Skip to content

Adding chat template support for Granite model #14864

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: master
Choose a base branch
from

Conversation

smdesai
Copy link

@smdesai smdesai commented Jul 24, 2025

This adds support for the chat template Granite model. It also modified build-xcframework.sh to include libcommon.a and supporting header files so the built framework can be used by a Swift wrapper for template processing.

@github-actions github-actions bot added the testing Everything test related label Jul 24, 2025
@ggerganov
Copy link
Member

It also modified build-xcframework.sh to include libcommon.a and supporting header files so the built framework can be used by a Swift wrapper for template processing.

The libcommon is not supposed to be used externally - it's only used by llama.cpp examples and tools.

@smdesai
Copy link
Author

smdesai commented Jul 28, 2025

Thank's for the update. I've reverted build-xcframework.sh to it's original form and the rest of the change set is model template specific which should be fine.

Copy link
Member

@ggerganov ggerganov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, though would be nice to have a second look from somebody more familiar with tools/templates.

@ggerganov ggerganov requested review from ochafik and Copilot July 28, 2025 05:41
Copy link

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR adds support for the Granite chat template format by IBM, including thinking (reasoning) capabilities and tool calling functionality. The implementation includes format detection, template processing, and message parsing for the Granite model.

  • Adds COMMON_CHAT_FORMAT_GRANITE enum value and associated parsing logic
  • Implements granite-specific thinking tag support with COMMON_REASONING_FORMAT_GRANITE
  • Enhances tool call argument parsing to handle both object and string formats

Reviewed Changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
common/common.h Adds COMMON_REASONING_FORMAT_GRANITE enum for granite thinking format
common/chat.h Adds COMMON_CHAT_FORMAT_GRANITE enum for granite chat format
common/chat.cpp Implements granite template initialization, parsing logic, and format detection
common/chat-parser.cpp Enhances tool call argument handling for object/string formats
tests/test-chat.cpp Adds comprehensive test coverage for granite format functionality
Comments suppressed due to low confidence (2)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
testing Everything test related
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants