Skip to content

Conversation

wu-hui
Copy link
Contributor

@wu-hui wu-hui commented Sep 9, 2025

No description provided.

Copy link
Contributor

github-actions bot commented Sep 9, 2025

Firebase AI Mock Responses Check ⚠️

A newer major version of the mock responses for Firebase AI unit tests is available. update_responses.sh should be updated to clone the latest version of the responses: v14.9

@wu-hui wu-hui changed the title Pipeline utils [realppl 2] Utils Sep 9, 2025
@google-oss-bot
Copy link
Contributor

1 Warning
⚠️ Did you forget to add a changelog entry? (Add the 'no-changelog' label to the PR to silence this warning.)

Generated by 🚫 Danger

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @wu-hui, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors and enhances the Firestore query pipeline's internal architecture. The changes focus on improving the canonical representation and evaluation capabilities of queries, especially for offline processing. By standardizing how pipeline components are identified and compared, and by streamlining the document processing flow, these updates contribute to a more robust, testable, and efficient query execution within the SDK.

Highlights

  • Pipeline Canonicalization: Introduced a new Canonicalizable interface and implemented it across key pipeline components (RealtimePipeline, Expr, and various Stage subclasses). This provides a consistent string representation for pipeline stages and expressions, enabling reliable comparison and hashing of pipeline objects.
  • Enhanced Pipeline Evaluation: The RealtimePipeline class now includes direct methods for evaluation such as evaluate(), matchesAllDocuments(), hasLimit(), matches(), and comparator(). This centralizes the logic for processing documents and determining query characteristics directly within the pipeline object.
  • Internal API Refinement: The QueryOrPipeline and TargetOrPipeline classes have been refactored into Kotlin sealed class hierarchies. This improves type safety and readability by explicitly defining the types of objects they can wrap, leveraging Kotlin's idiomatic features.
  • Shift to List-based Evaluation: The internal pipeline evaluation mechanism has transitioned from using Kotlin Flow (reactive streams) to direct List processing for document inputs and outputs in Stage.evaluate methods. This simplifies the evaluation logic, particularly for offline scenarios.
  • Improved Equality and Hashing: Comprehensive overrides for equals() and hashCode() methods have been added to numerous pipeline components. This ensures correct behavior when comparing and storing pipeline objects in collections, which is crucial for caching and deduplication.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in issue comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a significant and well-executed refactoring of the pipeline functionality, primarily to support local evaluation. The introduction of the Canonicalizable interface and the implementation of equals/hashCode across Stage and Expr classes are excellent changes that enhance correctness and maintainability. The switch to using sealed classes for QueryOrPipeline and TargetOrPipeline is also a great improvement.

I've found one critical bug that could cause a crash during local pipeline evaluation and a couple of medium-severity suggestions to improve code clarity and robustness. Overall, this is a high-quality contribution.

buffer.forEach { emit(it) }
}
else -> emptyFlow()
limit < 0 -> inputs.takeLast(limit)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The evaluate function for LimitStage has a bug when limit is negative. inputs.takeLast(limit) will throw an IllegalArgumentException because limit is negative, but takeLast expects a non-negative count. The intention is likely to take the last -limit elements.

Suggested change
limit < 0 -> inputs.takeLast(limit)
limit < 0 -> inputs.takeLast(-limit)

Comment on lines +841 to +843
if (stage.name == "limit") {
return false
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using stage.name == "limit" to check for the LimitStage is a bit brittle. It's better to use a type check with is LimitStage for improved type safety and maintainability. This avoids potential issues if the stage name is ever changed and makes the code's intent clearer.

A similar issue exists in the hasLimit() function on line 865.

Suggested change
if (stage.name == "limit") {
return false
}
if (stage is LimitStage) {
return false
}

Comment on lines +879 to +884
private fun evaluateContext(): EvaluationContext {
return EvaluationContext(this)
}

internal fun comparator(): Comparator<Document> =
getLastEffectiveSortStage().comparator(evaluateContext())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The evaluateContext() function is a private helper that is only used once. It can be inlined at its call site in comparator() to simplify the code and improve readability.

  internal fun comparator(): Comparator<Document> =
    getLastEffectiveSortStage().comparator(EvaluationContext(this))

@google-oss-bot
Copy link
Contributor

The public api surface has changed for the subproject firebase-firestore:
error: Added method com.google.firebase.firestore.FirebaseFirestore.realtimePipeline() [AddedMethod]
error: Added method com.google.firebase.firestore.Pipeline.execute(com.google.firebase.firestore.pipeline.PipelineOptions) [AddedMethod]
error: Removed method com.google.firebase.firestore.Pipeline.execute(com.google.firebase.firestore.pipeline.RealtimePipelineOptions) [RemovedMethod]
error: Added method com.google.firebase.firestore.RealtimePipeline.canonicalId() [AddedMethod]
error: Removed method com.google.firebase.firestore.RealtimePipeline.execute() [RemovedMethod]
error: Removed method com.google.firebase.firestore.RealtimePipeline.execute(com.google.firebase.firestore.pipeline.PipelineOptions) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.AbstractOptions.with(String,com.google.firebase.firestore.pipeline.GenericOptions) [RemovedMethod]
error: Added method com.google.firebase.firestore.pipeline.AbstractOptions.with(String,com.google.firebase.firestore.pipeline.RawOptions) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionGroupSource.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionGroupSource.getCollectionId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionSource.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionSource.getPath() [AddedMethod]
error: Removed class com.google.firebase.firestore.pipeline.ExplainOptions [RemovedClass]
error: Added method com.google.firebase.firestore.pipeline.Expr.mapGet(com.google.firebase.firestore.pipeline.Expr) [AddedMethod]
error: Attempted to change parameter name from key to keyExpression in method com.google.firebase.firestore.pipeline.Expr.mapRemove [ParameterNameChange]
error: Added method com.google.firebase.firestore.pipeline.ExprWithAlias.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.ExprWithAlias.getAlias() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.ExprWithAlias.getExpr() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Field.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.FunctionExpr.canonicalId() [AddedMethod]
error: Removed class com.google.firebase.firestore.pipeline.GenericOptions [RemovedClass]
error: Added method com.google.firebase.firestore.pipeline.Ordering.canonicalId() [AddedMethod]
error: Removed method com.google.firebase.firestore.pipeline.PipelineOptions.withExplainOptions(com.google.firebase.firestore.pipeline.ExplainOptions) [RemovedMethod]
error: Added class com.google.firebase.firestore.pipeline.RawOptions [AddedClass]
error: Removed method com.google.firebase.firestore.pipeline.Stage.getName() [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,String) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,boolean) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,com.google.firebase.firestore.pipeline.Field) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,double) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,error.NonExistentClass) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,long) [RemovedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,String) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,boolean) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,com.google.firebase.firestore.pipeline.Field) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,double) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,error.NonExistentClass) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,long) [AddedMethod]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

Copy link
Contributor

github-actions bot commented Sep 9, 2025

Test Results

0 tests  ±0   0 ✅ ±0   0s ⏱️ ±0s
0 suites ±0   0 💤 ±0 
0 files   ±0   0 ❌ ±0 

Results for commit 76763e4. ± Comparison against base commit 537313a.

♻️ This comment has been updated with latest results.

@google-oss-bot
Copy link
Contributor

google-oss-bot commented Sep 9, 2025

Coverage Report 1

Affected Products

No changes between base commit (537313a) and merge commit (cffd3d5).

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/T90Xh0ghU4.html

@google-oss-bot
Copy link
Contributor

Size Report 1

Affected Products

  • base

    TypeBase (350843c)Merge (9b0f635)Diff
    apk (aggressive)?8.80 kB? (?)
    apk (release)?9.77 kB? (?)
  • firebase-firestore

    TypeBase (350843c)Merge (9b0f635)Diff
    aar?2.13 MB? (?)
    apk (aggressive)?559 kB? (?)
    apk (release)?12.1 MB? (?)
  • protolite-well-known-types

    TypeBase (350843c)Merge (9b0f635)Diff
    aar?1.02 MB? (?)
    apk (aggressive)?134 kB? (?)
    apk (release)?1.71 MB? (?)

Test Logs

  1. https://storage.googleapis.com/firebase-sdk-metric-reports/u5MvAigxPZ.html

@google-oss-bot
Copy link
Contributor

The public api surface has changed for the subproject firebase-firestore:
error: Added method com.google.firebase.firestore.FirebaseFirestore.realtimePipeline() [AddedMethod]
error: Added method com.google.firebase.firestore.Pipeline.execute(com.google.firebase.firestore.pipeline.PipelineOptions) [AddedMethod]
error: Removed method com.google.firebase.firestore.Pipeline.execute(com.google.firebase.firestore.pipeline.RealtimePipelineOptions) [RemovedMethod]
error: Added method com.google.firebase.firestore.RealtimePipeline.canonicalId() [AddedMethod]
error: Removed method com.google.firebase.firestore.RealtimePipeline.execute() [RemovedMethod]
error: Removed method com.google.firebase.firestore.RealtimePipeline.execute(com.google.firebase.firestore.pipeline.PipelineOptions) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.AbstractOptions.with(String,com.google.firebase.firestore.pipeline.GenericOptions) [RemovedMethod]
error: Added method com.google.firebase.firestore.pipeline.AbstractOptions.with(String,com.google.firebase.firestore.pipeline.RawOptions) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionGroupSource.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionGroupSource.getCollectionId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionSource.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionSource.getPath() [AddedMethod]
error: Removed class com.google.firebase.firestore.pipeline.ExplainOptions [RemovedClass]
error: Added method com.google.firebase.firestore.pipeline.Expr.mapGet(com.google.firebase.firestore.pipeline.Expr) [AddedMethod]
error: Attempted to change parameter name from key to keyExpression in method com.google.firebase.firestore.pipeline.Expr.mapRemove [ParameterNameChange]
error: Added method com.google.firebase.firestore.pipeline.ExprWithAlias.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.ExprWithAlias.getAlias() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.ExprWithAlias.getExpr() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Field.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.FunctionExpr.canonicalId() [AddedMethod]
error: Removed class com.google.firebase.firestore.pipeline.GenericOptions [RemovedClass]
error: Added method com.google.firebase.firestore.pipeline.Ordering.canonicalId() [AddedMethod]
error: Removed method com.google.firebase.firestore.pipeline.PipelineOptions.withExplainOptions(com.google.firebase.firestore.pipeline.ExplainOptions) [RemovedMethod]
error: Added class com.google.firebase.firestore.pipeline.RawOptions [AddedClass]
error: Removed method com.google.firebase.firestore.pipeline.Stage.getName() [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,String) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,boolean) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,com.google.firebase.firestore.pipeline.Field) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,double) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,error.NonExistentClass) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,long) [RemovedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,String) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,boolean) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,com.google.firebase.firestore.pipeline.Field) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,double) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,error.NonExistentClass) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,long) [AddedMethod]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

1 similar comment
@google-oss-bot
Copy link
Contributor

The public api surface has changed for the subproject firebase-firestore:
error: Added method com.google.firebase.firestore.FirebaseFirestore.realtimePipeline() [AddedMethod]
error: Added method com.google.firebase.firestore.Pipeline.execute(com.google.firebase.firestore.pipeline.PipelineOptions) [AddedMethod]
error: Removed method com.google.firebase.firestore.Pipeline.execute(com.google.firebase.firestore.pipeline.RealtimePipelineOptions) [RemovedMethod]
error: Added method com.google.firebase.firestore.RealtimePipeline.canonicalId() [AddedMethod]
error: Removed method com.google.firebase.firestore.RealtimePipeline.execute() [RemovedMethod]
error: Removed method com.google.firebase.firestore.RealtimePipeline.execute(com.google.firebase.firestore.pipeline.PipelineOptions) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.AbstractOptions.with(String,com.google.firebase.firestore.pipeline.GenericOptions) [RemovedMethod]
error: Added method com.google.firebase.firestore.pipeline.AbstractOptions.with(String,com.google.firebase.firestore.pipeline.RawOptions) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionGroupSource.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionGroupSource.getCollectionId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionSource.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.CollectionSource.getPath() [AddedMethod]
error: Removed class com.google.firebase.firestore.pipeline.ExplainOptions [RemovedClass]
error: Added method com.google.firebase.firestore.pipeline.Expr.mapGet(com.google.firebase.firestore.pipeline.Expr) [AddedMethod]
error: Attempted to change parameter name from key to keyExpression in method com.google.firebase.firestore.pipeline.Expr.mapRemove [ParameterNameChange]
error: Added method com.google.firebase.firestore.pipeline.ExprWithAlias.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.ExprWithAlias.getAlias() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.ExprWithAlias.getExpr() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Field.canonicalId() [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.FunctionExpr.canonicalId() [AddedMethod]
error: Removed class com.google.firebase.firestore.pipeline.GenericOptions [RemovedClass]
error: Added method com.google.firebase.firestore.pipeline.Ordering.canonicalId() [AddedMethod]
error: Removed method com.google.firebase.firestore.pipeline.PipelineOptions.withExplainOptions(com.google.firebase.firestore.pipeline.ExplainOptions) [RemovedMethod]
error: Added class com.google.firebase.firestore.pipeline.RawOptions [AddedClass]
error: Removed method com.google.firebase.firestore.pipeline.Stage.getName() [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,String) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,boolean) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,com.google.firebase.firestore.pipeline.Field) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,double) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,error.NonExistentClass) [RemovedMethod]
error: Removed method com.google.firebase.firestore.pipeline.Stage.with(String,long) [RemovedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,String) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,boolean) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,com.google.firebase.firestore.pipeline.Field) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,double) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,error.NonExistentClass) [AddedMethod]
error: Added method com.google.firebase.firestore.pipeline.Stage.withOption(String,long) [AddedMethod]

Please update the api.txt files for the subprojects being affected by this change by running ./gradlew ${subproject}:generateApiTxtFile. Also perform a major/minor bump accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants