-
Notifications
You must be signed in to change notification settings - Fork 6.6k
fix: prevent empty tool names from overwriting valid names in SSE deltas #7633
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
All contributors have signed the CLA ✍️ ✅ |
|
I have read the CLA Document and I hereby sign the CLA |
This commit improves handling of malformed tool calls in SSE streams: 1. Defensive fix: Skip tool calls that have empty or missing names to prevent API errors downstream 2. Debug logging: Add detailed logging when skipping tool calls to help diagnose root cause (model issue vs streaming issue) 3. Test coverage: Add test case to verify empty/missing names are properly filtered out The issue can occur when: - The SSE stream from the API is incomplete or corrupted - The model sends tool call deltas without including the name field - Tool call streaming completes with finish_reason="tool_calls" but some tool calls never received their name in any delta The debug logs will show: - call_id of the problematic tool call - arguments that were received (if any) - whether the name was empty string or completely missing This helps identify if the issue is: 1. A model bug (consistently missing names for certain patterns) 2. A streaming issue (names lost during transmission) 3. A timing issue (finish_reason sent before all deltas arrived)
This commit addresses the root cause identified in GitHub issues: - openai#7094: Empty function name cause Azure to reject the request - openai#7517: Tool call streaming broken in 0.64.0 with local LLM providers - openai#7579: Regression: Tool Name Lost in Streaming Chat Completions Root Cause (Issue openai#7579): The SSE streaming parser was unconditionally overwriting tool call names with values from subsequent deltas. Many providers (Azure OpenAI, LM Studio, etc.) send the tool name in the first delta, then send empty strings in subsequent deltas containing only arguments. Before this fix: 1. First delta: name="my_tool", arguments="{" 2. Second delta: name="", arguments="foo}" 3. Result: name gets overwritten with "" -> causes API errors The Fix (Two Layers): 1. ROOT CAUSE FIX (lines 165-170): Don't overwrite existing tool call names with empty strings during delta parsing. Only update the name if the new value is non-empty. This preserves the name from the first delta. 2. DEFENSIVE FIX (lines 228-251): Even with the root cause fixed, we add defensive handling at the emission point to skip any tool calls that somehow end up with empty/missing names, with debug logging to help diagnose issues. Test Coverage: - preserves_name_when_subsequent_deltas_have_empty_names: Tests the root cause scenario (name in first delta, empty in subsequent) - skips_tool_calls_with_empty_or_missing_names: Tests the defensive layer for edge cases This fixes compatibility with: - Azure OpenAI (via LiteLLM) - LM Studio - Other OpenAI-compatible local providers
e614f9b to
211b5fc
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. You're on a roll. ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Thanks for the PR, but I think we'll pass on this one. |
Summary
Adds defensive handling and additional test coverage for empty tool call function names in SSE streaming, complementing the root cause fix in PR #7594.
Context
PR #7594 already fixed the root cause by using
get_or_insert_with()to prevent overwriting tool names. This PR builds on that fix by adding:Issues Addressed
Changes in This PR
1. Enhanced Documentation (lines 164-166)
Adds comment explaining the root cause and linking to issue #7579:
2. Defensive Layer at Emission Point (lines 228-251)
Even though the root cause is fixed, this adds defensive handling to skip any tool calls that somehow end up with empty/missing names, with detailed debug logging for troubleshooting:
This helps diagnose issues by logging:
call_idof problematic tool callsargumentsthat were received3. Additional Test Coverage
Adds
skips_tool_calls_with_empty_or_missing_namestest to verify the defensive layer works correctly when tool calls have empty or missing names.Test Results
All tests passing (7/7):
emits_multiple_tool_callsconcatenates_tool_call_arguments_across_deltasemits_tool_calls_even_when_content_and_reasoning_presentdrops_partial_tool_calls_on_stop_finish_reasonpreserves_tool_call_name_when_empty_deltas_arrive(from PR fix: sse for chat #7594)preserves_name_when_subsequent_deltas_have_empty_names(this PR)skips_tool_calls_with_empty_or_missing_names(this PR)Value Add
While PR #7594 fixed the root cause, this PR provides:
Testing
cargo test --package codex-api --lib sse::chat::tests