refactor: Utilize processor.yield_exceptions_as_parts in call

[mbeacom]

This pull request refactors the UrlFetchProcessor and its associated tests to replace failure parts with exception parts, leveraging a decorator to handle exceptions more consistently. It improves the clarity of error handling, simplifies the code, and enhances test coverage for various scenarios.

Refactor to Exception Handling in UrlFetchProcessor

• Removed _create_failure_part method and replaced its functionality with the @processor.yield_exceptions_as_parts decorator, which automatically converts exceptions into error parts. (genai_processors_url_fetch/url_fetch.py)
• Updated the async def call method to raise exceptions for failed fetches. Metadata is attached to exceptions for better error part generation. (genai_processors_url_fetch/url_fetch.py)

Updates to Test Cases

• Refactored test cases to validate exception parts instead of failure parts, ensuring consistent handling across various scenarios (e.g., failed fetches, URL validation errors, and response size limits). (genai_processors_url_fetch/tests/test_url_fetch.py)
  • Enhanced assertions for status and exception parts in test_failed_fetch_with_mocking.
  • Updated test_fail_on_error_config to verify exception conversion via the decorator.
  • Replaced failure parts with exception parts in test_url_validation_integration.
  • Adjusted test_response_size_limits and mock_aiter_bytes tests to check exception parts for size-related errors. [1] [2]

Minor Improvements

• Added a status message to indicate the processing of URLs in the call method. (genai_processors_url_fetch/url_fetch.py)

[gemini-code-assist]

Summary of Changes

Hello @mbeacom, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces a significant refactoring to the UrlFetchProcessor by standardizing its error handling mechanism. It transitions from a custom 'failure part' system to a more robust, decorator-driven approach that automatically converts exceptions into structured output parts. This change simplifies the core logic for handling failed URL fetches, enhances the clarity and consistency of error reporting, and is fully supported by comprehensive updates to the existing test suite.

Highlights

• Error Handling Refactor: The UrlFetchProcessor now leverages the @processor.yield_exceptions_as_parts decorator to automatically convert exceptions raised during URL fetching into standardized exception parts. This change replaces the manual creation of 'failure parts' and streamlines error reporting.
• Simplified call Method Logic: The async def call method has been simplified. Instead of explicitly creating failure parts or conditionally raising exceptions based on a fail_on_error configuration, it now consistently raises exceptions for failed fetches, relying on the new decorator to handle their conversion into output parts. Additionally, URL processing within call has shifted from concurrent fetching to sequential iteration, which better aligns with the per-exception yielding mechanism.
• Test Suite Updates: The test suite (test_url_fetch.py) has been thoroughly updated to reflect these changes. Tests now assert the presence and content of 'exception parts' instead of 'failure parts' for various error scenarios, including failed fetches, URL validation errors, and response size limits. A specific test (test_fail_on_error_config) was also updated to verify the decorator's behavior.
• Status Message Improvement: A new status message is now yielded at the beginning of the call method, providing clearer feedback on the number of URLs being processed.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist is currently in preview and may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments to provide feedback.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[Copilot]

Pull Request Overview

This pull request refactors the UrlFetchProcessor to replace manual failure part creation with exception-based error handling using the @processor.yield_exceptions_as_parts decorator. This change simplifies error handling by leveraging automatic exception-to-error-part conversion and removes the fail_on_error configuration logic since exceptions are now always raised for failed fetches.

• Replaces _create_failure_part method with decorator-based exception handling
• Changes from concurrent URL processing to sequential processing
• Updates all test cases to validate exception parts instead of failure parts

Reviewed Changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

File	Description
genai_processors_url_fetch/url_fetch.py	Removes failure part creation, adds decorator, changes to sequential processing, and always raises exceptions for failed fetches
genai_processors_url_fetch/tests/test_url_fetch.py	Updates test assertions to check for exception parts instead of failure parts and adjusts status part count expectations

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.34%. Comparing base (f7df057) to head (1a7eeb6).

Additional details and impacted files

@@            Coverage Diff             @@
##             main      #10      +/-   ##
==========================================
- Coverage   92.41%   92.34%   -0.07%     
==========================================
  Files           1        1              
  Lines         224      222       -2     
  Branches       34       33       -1     
==========================================
- Hits          207      205       -2     
  Misses         10       10              
  Partials        7        7              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[gemini-code-assist]

Code Review

This pull request successfully refactors the UrlFetchProcessor to use a decorator for exception handling, which simplifies the error logic. My review focuses on a performance regression where URL fetching was changed from concurrent to sequential, and I've provided suggestions to restore concurrency. I've also pointed out several areas in the tests that could be improved for clarity and completeness.