-
Notifications
You must be signed in to change notification settings - Fork 6
Fixes #28
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Fixes #28
Conversation
OpenAI wants only alpha-numeric characters, underscores or dashes. Nothing else.
structured output and function calling
Implements a comprehensive mode enumeration system similar to Python instructor, providing a centralized way to manage different API interaction modes across OpenAI and Anthropic providers. Features: - New Instructor::Mode module with OpenAI modes (TOOLS, TOOLS_STRICT, PARALLEL_TOOLS, JSON, JSON_SCHEMA, MD_JSON, FUNCTIONS) - Anthropic modes (ANTHROPIC_TOOLS, ANTHROPIC_JSON, ANTHROPIC_PARALLEL_TOOLS, ANTHROPIC_REASONING_TOOLS) - Classification methods: tool_modes, json_modes, tool_mode?, json_mode? - Mode validation with validate_mode! for early error detection - Deprecation warning system for FUNCTIONS mode - Backward compatibility with legacy :structured_output and :function_calling symbols Changes: - Created lib/instructor/mode.rb with unified mode constants - Updated lib/instructor.rb to use TOOLS_STRICT as default mode - Deprecated Instructor::OpenAI::Mode in favor of unified Instructor::Mode - Enhanced lib/instructor/openai/patch.rb with mode detection helpers - Enhanced lib/instructor/openai/response.rb with mode-based response handling - Added Response.create factory method to Anthropic::Response for consistency - Updated all tests to use new mode system with backward compatibility tests - All 99 examples passing with full backward compatibility
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Caution
Changes requested ❌
Reviewed everything up to 6c09306 in 2 minutes and 17 seconds. Click for details.
- Reviewed
4445
lines of code in40
files - Skipped
1
files when reviewing. - Skipped posting
11
draft comments. View those below. - Modify your settings and rules to customize what types of comments Ellipsis leaves. And don't forget to react with 👍 or 👎 to teach Ellipsis.
1. lib/instructor/mode.rb:63
- Draft comment:
The use of Set[...] is good; however, ensure that the 'set' library is required so that Set is available in all environments. - Reason this comment was not posted:
Confidence changes required:50%
<= threshold50%
None
2. lib/instructor/openai/patch.rb:30
- Draft comment:
In the build_function method, the constructed hash nests the function details under a :function key. It is important to be consistent with the function payload format expected by the API. - Reason this comment was not posted:
Confidence changes required:30%
<= threshold50%
None
3. lib/instructor/openai/response.rb:94
- Draft comment:
The parse method in ToolResponse directly calls JSON.parse on the function response. Consider adding explicit error handling or validation of the function response structure before parsing. - Reason this comment was not posted:
Confidence changes required:40%
<= threshold50%
None
4. spec/instructor/mode_spec.rb:55
- Draft comment:
Tests here validate that tool_modes and json_modes return Set objects containing the correct mode symbols. Looks good, but consider also checking that no extraneous modes are included. - Reason this comment was not posted:
Confidence changes required:20%
<= threshold50%
None
5. spec/openai/patch_spec.rb:81
- Draft comment:
The test for generating function names lowercases the model's title. Ensure that this behavior is intentional and documented, since this normalization could affect integration with systems expecting case-sensitive names. - Reason this comment was not posted:
Confidence changes required:30%
<= threshold50%
None
6. spec/instructor_spec.rb:23
- Draft comment:
The tests in this file support legacy mode symbols (:structured_output, :function_calling). Ensure proper deprecation warnings are emitted and documented for backward compatibility. - Reason this comment was not posted:
Confidence changes required:30%
<= threshold50%
None
7. lib/instructor/openai/patch.rb:124
- Draft comment:
Typo: The comment on line 124 uses "parameters's" which appears to be a mistake. Consider revising it to either "parameters'" or "parameter's" to clarify the intended meaning. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% While the comment is technically correct about the grammar, it's addressing a very minor issue in a code comment. The comment itself appears to be a TODO or placeholder comment that will likely be replaced with actual documentation. The rules state not to make purely informative comments or comments about obvious/unimportant issues. The grammar error could make the code documentation less professional, and unclear documentation can lead to confusion. However, this is an extremely minor issue that doesn't affect code functionality, and the comment itself looks like a placeholder that will be replaced anyway. The comment should be deleted as it addresses a trivial grammatical issue in what appears to be a placeholder comment.
8. spec/openai/patch_spec.rb:107
- Draft comment:
The model name 'gpt-4o-2024-08-06' looks unusual. Please confirm if 'gpt-4o' is intended or if it should be, for example, 'gpt-4' (or another correct variant). - Reason this comment was not posted:
Comment looked like it was already resolved.
9. spec/openai/patch_spec.rb:108
- Draft comment:
The message string 'Extract Jason is 25 years old' uses 'Jason'. If this is meant to refer to JSON or if it is a person's name, please verify that 'Jason' is correct. - Reason this comment was not posted:
Comment was on unchanged code.
10. spec/vcr_cassettes/features/openai/basic_spec/valid_response.yml:131
- Draft comment:
Typographical issue: The 'recorded_with' key appears to be mis‐indented relative to 'recorded_at'. It should likely be indented consistently with the surrounding keys. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 0% vs. threshold = 50% VCR cassettes are automatically generated files. The indentation is controlled by VCR's serialization logic, not by developers. Making manual formatting changes to auto-generated files is generally not recommended as they'll just be overwritten next time the tests run. Additionally, the indentation doesn't affect the functionality of the VCR cassette. Maybe there's actually a bug in VCR's YAML generation that should be fixed upstream? Maybe inconsistent indentation could cause issues with some YAML parsers? Even if there was a VCR bug, fixing indentation manually in the generated file isn't the solution - it would need to be fixed in VCR itself. The current indentation works fine with YAML parsers as evidenced by the tests passing. Delete this comment as it's suggesting manual changes to an auto-generated file, which is not a useful suggestion.
11. spec/vcr_cassettes/iterable_spec/valid_response.yml:10
- Draft comment:
Note: There's a potential inconsistency in the function name's casing. The "name" field is now set to "users" (lowercase) while the description still refers toUsers
(capitalized). Please confirm if this is intentional. - Reason this comment was not posted:
Decided after close inspection that this draft comment was likely wrong and/or not actionable: usefulness confidence = 10% vs. threshold = 50% This is a test fixture file that records API interactions. The casing inconsistency is minor and doesn't affect functionality. The comment is asking for confirmation ("Please confirm if this is intentional") which violates our rules about not asking for confirmations. Additionally, this is the kind of minor detail that doesn't require a code change. Maybe the casing inconsistency could cause confusion for other developers reading the code? Maybe it indicates a deeper issue with naming conventions? While consistency is good, this is in a test fixture file that just records API interactions. The casing difference is too minor to warrant a comment, and the actual code using this fixture will work regardless. Delete the comment. It asks for confirmation about a minor casing inconsistency that doesn't affect functionality, especially in a test fixture file.
Workflow ID: wflow_yt2gGyH1YzPJYWDB
You can customize by changing your verbosity settings, reacting with 👍 or 👎, replying to comments, or adding code review rules.
# @param mode [Symbol] The mode to check | ||
# @return [Boolean] true if mode uses tools | ||
def tool_mode?(mode) | ||
Instructor::Mode.tool_mode?(mode) && mode.to_s.start_with?('anthropic') |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The method tool_mode? uses mode.to_s.start_with!("anthropic") to decide if the mode is tool‐based. This string-based check is a bit brittle. Consider comparing against a set of known Anthropic mode symbols for more clarity.
Instructor::Mode.tool_mode?(mode) && mode.to_s.start_with?('anthropic') | |
Instructor::Mode.tool_mode?(mode) && [Instructor::Mode::ANTHROPIC_TOOLS, Instructor::Mode::ANTHROPIC_REASONING_TOOLS, Instructor::Mode::ANTHROPIC_PARALLEL_TOOLS].include?(mode) |
end | ||
|
||
it 'raises an error when the response model is invalid', vcr: 'patching_spec/invalid_response' do | ||
it 'raises an argument error when the model resfuses to respond', vcr: 'openai/patch/invalid_response' do |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Typo: "resfuses" appears to be a misspelling. It likely should be "refuses".
it 'raises an argument error when the model resfuses to respond', vcr: 'openai/patch/invalid_response' do | |
it 'raises an argument error when the model refuses to respond', vcr: 'openai/patch/invalid_response' do |
Important
Enhance
Instructor
library with updated dependencies, new modes for API interactions, and improved response handling, along with comprehensive test coverage.activesupport
to>= 6.0
.easy_talk
to~> 2
.ruby-anthropic
to~> 0.4
.ruby-openai
to~> 8
.ANTHROPIC_TOOLS
,ANTHROPIC_JSON
,ANTHROPIC_PARALLEL_TOOLS
, andANTHROPIC_REASONING_TOOLS
modes inmode.rb
.TOOLS_STRICT
,JSON_SCHEMA
, andMD_JSON
modes for OpenAI inmode.rb
.from_openai
andfrom_anthropic
methods ininstructor.rb
to support new modes.Anthropic::Patch
andOpenAI::Patch
to handle new modes and response formats.ToolResponse
andJsonResponse
classes inanthropic/response.rb
andopenai/response.rb
for mode-specific response handling.spec/anthropic
andspec/openai
directories.This description was created by
for 6c09306. You can customize this summary. It will automatically update as commits are pushed.