Implement move-in/out algorithm in Elixir Client #3674
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This plan outlines the changes needed to add subquery move support: - Phase 1: Create TagIndex and MoveState modules for tag tracking - Phase 2: Update message types (EventMessage, tags in Headers) - Phase 3: Update stream processing to handle move-outs - Phase 4: Add comprehensive tests - Phase 5: Documentation updates The implementation follows the same algorithm as TanStack DB client, using a positional tag index for efficient pattern matching.
Formatted issue with: - Background and wire protocol examples - Phased implementation plan with code snippets - File change summary - Acceptance criteria checklist - Design decisions documentation
This adds support for tagged rows and move-out events, enabling proper handling of shapes with subqueries in their WHERE clauses. New modules: - TagIndex: Positional index for efficient move-out pattern matching - MoveState: Tracks tags per row and handles move-out processing Message changes: - Headers: Added tags and removed_tags fields - EventMessage: New struct for move-out events - ControlMessage: Added xmin/xmax/xip_list for snapshot-end visibility Stream changes: - Track move_state for tag management - Buffer move-out events during initial sync - Process buffered move-outs on up-to-date - Generate synthetic delete messages for rows with empty tag sets The implementation follows the same algorithm as TanStack DB client, ensuring consistent behavior across all Electric client implementations.
✅ Deploy Preview for electric-next ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
❌ 1 Tests Failed:
View the top 1 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
This comment has been minimized.
This comment has been minimized.
Wildcards in tags are not indexed for efficiency (consistent with TanStack DB implementation). Updated test to document this expected behavior: - Pattern lookups at wildcard positions don't find rows - Pattern lookups at non-wildcard positions still work correctly
Tests cover: - Change messages with tags (insert, update, delete) - Move-out events generating synthetic deletes - Rows with remaining tags not being deleted - Multiple rows matching move-out patterns - Buffering move-outs during initial sync - Message parsing for events, tags, and snapshot-end - Stream state reset on must-refetch
This reverts commit 8b3411a.
This comment has been minimized.
This comment has been minimized.
balegas
changed the title
Tests use real Postgres and Electric server to verify: - Shapes with subquery WHERE clauses receive tagged rows - Move-out events triggered when parent rows no longer match - Move-in events when parent rows start matching - Synthetic deletes generated for rows with emptied tag sets - Multiple children affected by single parent move-out Test scenarios mirror sync-service router tests for subqueries.
Contributor
|
Found 1 test failure on Blacksmith runners: Failure
|
- Use local path dependency for electric to ensure feature_flags flow correctly from runtime.exs through StackSupervisor to StackConfig - Enable allow_subqueries and tagged_subqueries feature flags for tests - Update tests to expect synthetic DELETE messages instead of raw EventMessage (Stream module processes move-out events internally) - Remove debug code and unused functions from test file 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #3672