[ENHANCEMENT] OpenAI-compatible: Separate handlers for Chat Completions vs Responses API (maintainable architecture)

[hannesrudolph]

Type

Enhancement

Problem / Value

Adding Responses API support alongside Chat Completions in a single handler becomes hard to maintain. The two APIs differ significantly (params, streaming formats, response handling). Merging both into openai.ts would create complexity and long-term maintenance risk.

Context

• Responses API introduces many specialized streaming events (30+)
• openai-native.ts already has ~1000 lines of dedicated Responses logic
• A combined approach in OpenAI Compatible would quickly become tangled

Proposal

Introduce clear separation with a small shared core:

1. A base class with shared OpenAI logic (auth, retry, metrics, basic transforms)
2. Separate handlers for:
   • Chat Completions
   • Responses API (specialized streaming/response parsing)
3. Auto-detect via URL pattern and route to the correct handler

This mirrors the successful pattern in openai-native.ts and keeps both paths maintainable as OpenAI evolves the Responses API or phases out Chat Completions.

Acceptance Criteria

• OpenAI-compatible provider routes requests based on URL pattern to either Chat Completions or Responses handlers
• Responses API handler supports its streaming/event model without leaking complexity into the Chat handler
• Shared base avoids duplication (auth, request wiring, error normalization)
• Clear, testable boundaries; adding new Responses events doesn’t impact Chat handler

Notes

• Use openai-native.ts as a reference design for Responses API support
• Keep file structure explicit (e.g., openai-responses.ts) or refactor with a base + two concrete classes
• I’m happy to help with the implementation if this direction is approved

[hannesrudolph]

@daniel-lxs based this off #6862 (comment)

[roomote]

I'm exploring a potential fix for this enhancement to separate handlers for Chat Completions vs Responses API. I'll work on implementing the proposed architecture with a shared base class and separate handlers for better maintainability.

[roomote]

I've opened PR #8248 with the implementation for separating OpenAI Chat Completions and Responses API handlers.

The implementation includes:

• A base class with shared logic (auth, retry, metrics, error handling)
• Separate for Chat API
• Separate for Responses API
• Auto-routing in based on URL patterns and model types
• Full backward compatibility with existing code
• Comprehensive test coverage

All tests are passing locally and CI checks are currently running. The architecture review showed 95% confidence with all acceptance criteria met.

Feedback welcome!