Skip to content

Openai compatible models: "reasoning" block missing in AIMessageChunk, and AIMessage #34153

@mkc-cho

Description

@mkc-cho

Checked other resources

  • This is a feature request, not a bug report or usage question.
  • I added a clear and descriptive title that summarizes the feature request.
  • I used the GitHub search to find a similar feature request and didn't find it.
  • I checked the LangChain documentation and API reference to see if this feature already exists.
  • This is not related to the langchain-community package.

Package (Required)

  • langchain
  • langchain-openai
  • langchain-anthropic
  • langchain-classic
  • langchain-core
  • langchain-cli
  • langchain-model-profiles
  • langchain-tests
  • langchain-text-splitters
  • langchain-chroma
  • langchain-deepseek
  • langchain-exa
  • langchain-fireworks
  • langchain-groq
  • langchain-huggingface
  • langchain-mistralai
  • langchain-nomic
  • langchain-ollama
  • langchain-perplexity
  • langchain-prompty
  • langchain-qdrant
  • langchain-xai
  • Other / not sure / general

Feature Description

Add support for reasoning outputs from OpenAI-compatible models that don't support the Responses API.

Current behavior:

  • OpenAI-compatible servers that support reasoning but not Responses API have their reasoning content ignored
  • Reasoning content in choices[].message.reasoning or choices[].message.reasoning_content is not extracted

Expected behavior:

  • Extract reasoning content from standard Chat Completions API responses
  • Include reasoning in AIMessage.content or AIMessageChunk.content

Use Case

Some OpenAI-compatible inference servers (vLLM, custom deployments) support reasoning models but does not support Responses API.

Problem:

  • Enterprise self-hosted inference servers with reasoning models
    • For Example:
      • OpenAI Compatible API provided by vLLM
      • OpenAI Compatible API provided by Cohere (Cannot use langchain-cohere because of bug in it, I already reported)
      • Other providers that provide OpenAI Compatible API but does not support Responses API
  • Reasoning content is lost and inaccessible through LangChain
    • Must manually parse raw responses to extract reasoning
    • Breaks compatibility with agent frameworks

Example:

llm = ChatOpenAI(
    base_url="https://custom-server.com/v1",
    model="reasoning-model",
    reasoning_effort="medium", 
    use_responses_api=False  # Server doesn't support it
)

response = llm.invoke("Solve this problem...")
# Currently: reasoning is ignored
# Needed: reasoning accessible in response.content or response.content_blocks

Why needed:

  • Broader compatibility with OpenAI-compatible servers that implement reasoning
  • Many servers support Chat Completions API with reasoning fields but not the full Responses API
  • Enable reasoning capabilities across diverse inference server implementations

Proposed Solution

Expectation:

  1. Detect reasoning fields in API response:
    • Check choice.message.reasoning or choice.message.reasoning_content
    • Only when use_responses_api=False and reasoning_effort (or reasoning) is provided
  2. Include in message content:
    • Add as content block with type="reasoning"
    • Similar format to Responses API handling
  3. Maintain backward compatibility:
    • Only process when fields exist in response
    • No change for models without reasoning support

Alternatives Considered

No response

Additional Context

This could be similar to #34011 and #34014

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature requestrequest for an enhancement / additional functionalitylangchainRelated to the package `langchain`openai

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions