Skip to content

Conversation

@KyleAMathews
Copy link
Collaborator

@KyleAMathews KyleAMathews commented Oct 21, 2025

Summary

This PR introduces a new paced mutations system for TanStack DB, enabling optimistic mutations with pluggable timing strategies. This provides a powerful way to control when and how mutations are persisted to the backend.

Powered by TanStack Pacer, which provides the underlying debounce, throttle, and queue implementations.

paced-mutations-demo.mp4

Key Design

Debounce/Throttle: Only one pending transaction (collecting mutations) and one persisting transaction (writing to backend) at a time. Multiple rapid mutations automatically merge together.

Queue: Each mutation creates a separate transaction, guaranteed to run in the order they're made (FIFO by default, configurable to LIFO).

Key Features

  • Pluggable Strategy System: Debounce, queue, throttle strategies for controlling mutation timing
  • Auto-merging Mutations: Multiple rapid mutations on the same item automatically merge for efficiency (debounce/throttle only)
  • React Hook: usePacedMutations for easy integration in React apps
  • Transaction Management: Track mutation state (pending → persisting → completed/failed)
  • Interactive Demo: Slider-based demo showcasing all strategies with real-time visualization

Example Usage

Debounce Strategy (Auto-save)

import { usePacedMutations, debounceStrategy } from "@tanstack/react-db"

function AutoSaveForm() {
  const mutate = usePacedMutations({
    mutationFn: async ({ transaction }) => {
      await api.save(transaction.mutations)
    },
    strategy: debounceStrategy({ wait: 500 }),
  })

  const handleChange = (value: string) => {
    const tx = mutate(() => {
      formCollection.update(formId, (draft) => {
        draft.content = value
      })
    })

    // Optionally await persistence
    await tx.isPersisted.promise
  }

  return <textarea onChange={e => handleChange(e.target.value)} />
}

Throttle Strategy (Volume Slider)

import { usePacedMutations, throttleStrategy } from "@tanstack/react-db"

function VolumeSlider() {
  const mutate = usePacedMutations({
    mutationFn: async ({ transaction }) => {
      await api.updateVolume(transaction.mutations)
    },
    strategy: throttleStrategy({ wait: 200 }),
  })

  return (
    <input
      type="range"
      onChange={e => {
        mutate(() => {
          settingsCollection.update('volume', draft => {
            draft.value = +e.target.value
          })
        })
      }}
    />
  )
}

Queue Strategy (Sequential Processing)

import { usePacedMutations, queueStrategy } from "@tanstack/react-db"

function SequentialUpload() {
  const mutate = usePacedMutations({
    mutationFn: async ({ transaction }) => {
      await api.uploadImages(transaction.mutations)
    },
    // FIFO by default - each mutation creates a separate transaction
    // Waits 500ms between processing each transaction
    strategy: queueStrategy({ wait: 500 }),
  })

  const handleFileSelect = (files: FileList) => {
    // Each file creates its own transaction, queued for sequential processing
    Array.from(files).forEach((file, idx) => {
      mutate(() => {
        uploadCollection.insert({ id: idx, file })
      })
    })
  }

  return <input type="file" multiple onChange={e => handleFileSelect(e.target.files!)} />
}

Available Strategies

Strategy Behavior Guarantees
debounceStrategy Wait for inactivity before persisting. One pending + one persisting tx at a time. Only final state persists
throttleStrategy Ensure minimum spacing between executions. One pending + one persisting tx at a time. Mutations between executions are merged
queueStrategy Each mutation becomes a separate transaction, processed sequentially in order (FIFO by default, configurable to LIFO). All mutations guaranteed to persist in order

Core Implementation

Paced Mutations (packages/db/src/paced-mutations.ts)

  • Creates transactions with autoCommit: false for strategy control
  • Debounce/Throttle: Manages one ambient transaction for collecting mutations. New mutations merge into this transaction until it's persisted.
  • Queue: Creates a separate transaction per mutate() call. Each transaction is queued and processed in order.
  • Strategies control when commit() is called via callback

Strategies (packages/db/src/strategies/)

  • Debounce: Resets timer on each mutation, commits after inactivity (via TanStack Pacer)
  • Queue: Each mutation gets its own transaction, processes sequentially in FIFO order (configurable to LIFO) (via TanStack Pacer)
  • Throttle: Ensure minimum spacing between executions (via TanStack Pacer)

React Hook (packages/react-db/src/usePacedMutations.ts)

  • Memoized strategy and mutationFn to prevent unnecessary recreations
  • Returns stable mutate callback
  • Full TypeScript support with generics

Demo Application

Interactive demo at examples/react/paced-mutations-demo/ showcasing:

  • Slider interface that triggers mutations on every change (demonstrates rapid mutations naturally)
  • All three strategies with configurable options (300ms default)
  • Real-time transaction timeline visualization
  • Optimistic vs synced state comparison
  • Detailed timing metrics (pending time, persisting time, total duration)
  • Fake server sync to demonstrate realistic optimistic updates

Tests

Comprehensive test suite for all strategies:

  • ✅ Debounce: batching and timer reset behavior
  • ✅ Queue: sequential processing with separate transactions per mutation
  • ✅ Throttle: leading/trailing edge execution
  • ✅ 100% coverage on usePacedMutations hook
  • ✅ All tests passing

Bug Fixes

  • Fixed queue strategy to create separate transactions per mutation (clears activeTransaction after each)
  • Fixed queue strategy to properly commit captured transactions (prevents "no active transaction" errors)
  • Fixed queue strategy to default to FIFO order (addItemsTo='back', getItemsFrom='front')
  • Fixed slider value resetting to 0 after persistence (use mutation.modified instead of mutation.changes)

Use Cases

Debounce Strategy:

  • Auto-save forms
  • Search-as-you-type
  • Settings panels

Queue Strategy:

  • Sequential workflows (file uploads, multi-step processes)
  • Rate-limited APIs
  • Audit trails where every operation must be recorded

Throttle Strategy:

  • Analytics tracking
  • Progress updates
  • Volume/brightness sliders

🤖 Generated with Claude Code

KyleAMathews and others added 4 commits October 20, 2025 15:38
Implements a new hook for managing optimistic mutations with pluggable timing strategies (debounce, queue, throttle) using TanStack Pacer.

Key features:
- Auto-merge mutations and serialize persistence according to strategy
- Track and rollback superseded pending transactions to prevent memory leaks
- Proper cleanup of pending/executing transactions on unmount
- Queue strategy uses AsyncQueuer for true sequential processing

Breaking changes from initial design:
- Renamed from useSerializedTransaction to useSerializedMutations (more accurate name)
- Each mutate() call creates mutations that are auto-merged, not separate transactions

Addresses feedback:
- HIGH: Rollback superseded transactions to prevent orphaned isPersisted promises
- HIGH: cleanup() now properly rolls back all pending/executing transactions
- HIGH: Queue strategy properly serializes commits using AsyncQueuer with concurrency: 1

Example usage:
```tsx
const mutate = useSerializedMutations({
  mutationFn: async ({ transaction }) => {
    await api.save(transaction.mutations)
  },
  strategy: debounceStrategy({ wait: 500 })
})
```

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Fixes for feedback-4 issues:
- Queue strategy: await isPersisted.promise instead of calling commit() again to fix double-commit error
- cleanup(): check transaction state before rollback to prevent errors on completed transactions
- Pending transactions: rollback all pending transactions on each new mutate() call to handle dropped callbacks

Added interactive serialized mutations demo:
- Visual tracking of transaction states (pending/executing/completed/failed)
- Live configuration of debounce/queue/throttle strategies
- Real-time stats dashboard showing transaction counts
- Transaction timeline with mutation details and durations

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
      Core fixes:
      - Save transaction reference before calling strategy.execute() to prevent null returns when strategies (like queue) execute
      callbacks synchronously
      - Call strategy.execute() on every mutate() call to properly reset debounce timers
      - Simplified transaction lifecycle - single active transaction that gets reused for batching

      Demo improvements:
      - Memoized strategy and mutationFn to prevent unnecessary recreations
      - Added fake server sync to demonstrate optimistic updates
      - Enhanced UI to show optimistic vs synced state and detailed timing
      - Added mitt for event-based server communication

      Tests:
      - Replaced comprehensive test suite with focused debounce strategy tests
      - Two tests demonstrating batching and timer reset behavior
      - Tests pass with real timers and validate mutation auto-merging

      🤖 Generated with [Claude Code](https://claude.com/claude-code)
@changeset-bot
Copy link

changeset-bot bot commented Oct 21, 2025

🦋 Changeset detected

Latest commit: 7b4e55a

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 13 packages
Name Type
@tanstack/db Patch
@tanstack/react-db Patch
@tanstack/angular-db Patch
@tanstack/electric-db-collection Patch
@tanstack/query-db-collection Patch
@tanstack/rxdb-db-collection Patch
@tanstack/solid-db Patch
@tanstack/svelte-db Patch
@tanstack/trailbase-db-collection Patch
@tanstack/vue-db Patch
todos Patch
@tanstack/db-example-paced-mutations-demo Patch
@tanstack/db-example-react-todo Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@pkg-pr-new
Copy link

pkg-pr-new bot commented Oct 21, 2025

More templates

@tanstack/angular-db

npm i https://pkg.pr.new/@tanstack/angular-db@704

@tanstack/db

npm i https://pkg.pr.new/@tanstack/db@704

@tanstack/db-ivm

npm i https://pkg.pr.new/@tanstack/db-ivm@704

@tanstack/electric-db-collection

npm i https://pkg.pr.new/@tanstack/electric-db-collection@704

@tanstack/query-db-collection

npm i https://pkg.pr.new/@tanstack/query-db-collection@704

@tanstack/react-db

npm i https://pkg.pr.new/@tanstack/react-db@704

@tanstack/rxdb-db-collection

npm i https://pkg.pr.new/@tanstack/rxdb-db-collection@704

@tanstack/solid-db

npm i https://pkg.pr.new/@tanstack/solid-db@704

@tanstack/svelte-db

npm i https://pkg.pr.new/@tanstack/svelte-db@704

@tanstack/trailbase-db-collection

npm i https://pkg.pr.new/@tanstack/trailbase-db-collection@704

@tanstack/vue-db

npm i https://pkg.pr.new/@tanstack/vue-db@704

commit: 7b4e55a

@github-actions
Copy link
Contributor

github-actions bot commented Oct 21, 2025

Size Change: +1.53 kB (+1.81%)

Total Size: 85.8 kB

Filename Size Change
./packages/db/dist/esm/index.js 1.69 kB +69 B (+4.25%)
./packages/db/dist/esm/paced-mutations.js 528 B +528 B (new file) 🆕
./packages/db/dist/esm/strategies/debounceStrategy.js 248 B +248 B (new file) 🆕
./packages/db/dist/esm/strategies/queueStrategy.js 433 B +433 B (new file) 🆕
./packages/db/dist/esm/strategies/throttleStrategy.js 248 B +248 B (new file) 🆕
ℹ️ View Unchanged
Filename Size
./packages/db/dist/esm/collection/change-events.js 1.63 kB
./packages/db/dist/esm/collection/changes.js 1.01 kB
./packages/db/dist/esm/collection/events.js 413 B
./packages/db/dist/esm/collection/index.js 3.23 kB
./packages/db/dist/esm/collection/indexes.js 1.16 kB
./packages/db/dist/esm/collection/lifecycle.js 1.8 kB
./packages/db/dist/esm/collection/mutations.js 2.52 kB
./packages/db/dist/esm/collection/state.js 3.79 kB
./packages/db/dist/esm/collection/subscription.js 2.2 kB
./packages/db/dist/esm/collection/sync.js 2.2 kB
./packages/db/dist/esm/deferred.js 230 B
./packages/db/dist/esm/errors.js 3.48 kB
./packages/db/dist/esm/event-emitter.js 798 B
./packages/db/dist/esm/indexes/auto-index.js 794 B
./packages/db/dist/esm/indexes/base-index.js 835 B
./packages/db/dist/esm/indexes/btree-index.js 2 kB
./packages/db/dist/esm/indexes/lazy-index.js 1.21 kB
./packages/db/dist/esm/indexes/reverse-index.js 577 B
./packages/db/dist/esm/local-only.js 967 B
./packages/db/dist/esm/local-storage.js 2.4 kB
./packages/db/dist/esm/optimistic-action.js 294 B
./packages/db/dist/esm/proxy.js 3.86 kB
./packages/db/dist/esm/query/builder/functions.js 615 B
./packages/db/dist/esm/query/builder/index.js 4.04 kB
./packages/db/dist/esm/query/builder/ref-proxy.js 938 B
./packages/db/dist/esm/query/compiler/evaluators.js 1.55 kB
./packages/db/dist/esm/query/compiler/expressions.js 760 B
./packages/db/dist/esm/query/compiler/group-by.js 2.04 kB
./packages/db/dist/esm/query/compiler/index.js 2.21 kB
./packages/db/dist/esm/query/compiler/joins.js 2.65 kB
./packages/db/dist/esm/query/compiler/order-by.js 1.43 kB
./packages/db/dist/esm/query/compiler/select.js 1.28 kB
./packages/db/dist/esm/query/ir.js 785 B
./packages/db/dist/esm/query/live-query-collection.js 404 B
./packages/db/dist/esm/query/live/collection-config-builder.js 5.54 kB
./packages/db/dist/esm/query/live/collection-registry.js 233 B
./packages/db/dist/esm/query/live/collection-subscriber.js 2.11 kB
./packages/db/dist/esm/query/optimizer.js 3.26 kB
./packages/db/dist/esm/scheduler.js 1.29 kB
./packages/db/dist/esm/SortedMap.js 1.24 kB
./packages/db/dist/esm/transactions.js 3.05 kB
./packages/db/dist/esm/utils.js 1.01 kB
./packages/db/dist/esm/utils/browser-polyfills.js 365 B
./packages/db/dist/esm/utils/btree.js 6.01 kB
./packages/db/dist/esm/utils/comparison.js 754 B
./packages/db/dist/esm/utils/index-optimization.js 1.73 kB

compressed-size-action::db-package-size

Added test coverage for all three mutation strategies:
- Debounce: batching and timer reset (already passing)
- Queue: accumulation and sequential processing
- Throttle: leading/trailing edge execution

All 5 tests passing with 100% coverage on useSerializedMutations hook.

Also added changeset documenting the new serialized mutations feature.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@github-actions
Copy link
Contributor

github-actions bot commented Oct 21, 2025

Size Change: 0 B

Total Size: 2.89 kB

ℹ️ View Unchanged
Filename Size
./packages/react-db/dist/esm/index.js 168 B
./packages/react-db/dist/esm/useLiveInfiniteQuery.js 1.41 kB
./packages/react-db/dist/esm/useLiveQuery.js 1.31 kB

compressed-size-action::react-db-package-size

KyleAMathews and others added 8 commits October 21, 2025 18:38
…tests

Added non-null assertions and proper type casting for test variables
to satisfy TypeScript's strict null checking. All 62 tests still passing
with 100% coverage on useSerializedMutations hook.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Changed from button-based mutations to a slider interface that better
demonstrates the different strategies in action:

- Changed Item.value from string to number (was already being used as number)
- Reduced default wait time from 1000ms to 300ms for more responsive demo
- Replaced "Trigger Mutation" and "Trigger 5 Rapid Mutations" buttons with
  a slider (0-100 range) that triggers mutations on every change
- Updated UI text to reference slider instead of buttons
- Changed mutation display from "value X-1 → X" to "value = X" since slider
  sets absolute values rather than incrementing

The slider provides a more natural and vivid demonstration of how strategies
handle rapid mutations - users can drag it and see debounce wait for stops,
throttle sample during drags, and queue process all changes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
- Use mutation.modified instead of mutation.changes for updates to preserve full state
- Remove Delta stat card as it wasn't providing value
- Show newest transactions first in timeline for better UX

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Queue strategy now receives a closure that commits the captured transaction instead of calling commitCallback which expects activeTransaction to be set. This prevents "no active transaction exists" errors.

- Capture transaction before clearing activeTransaction for queue strategy
- Pass commit closure to queue that operates on captured transaction
- Remove "Reset to 0" button from demo
- All tests passing

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Set explicit defaults for addItemsTo='back' and getItemsFrom='front' to ensure queue strategy processes transactions in FIFO order (oldest first).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
…gurable order

Update changeset to reflect that queue strategy creates separate transactions per mutation and defaults to FIFO (but is configurable).

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Rename the feature from "Serialized Mutations" to "Paced Mutations" to better reflect its purpose of controlling mutation timing rather than serialization. This includes:

- Renamed core functions: createSerializedMutations → createPacedMutations
- Renamed React hook: useSerializedMutations → usePacedMutations
- Renamed types: SerializedMutationsConfig → PacedMutationsConfig
- Updated all file names, imports, exports, and documentation
- Updated demo app title and examples
- Updated changeset

All tests pass and the demo app builds successfully.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
@KyleAMathews KyleAMathews changed the title feat: Add serialized mutations with timing strategies feat: Add paced mutations with timing strategies Oct 22, 2025
KyleAMathews and others added 3 commits October 22, 2025 10:46
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Update todo example and queueStrategy JSDoc to use usePacedMutations instead of useSerializedMutations.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add reference to TanStack Pacer which powers the paced mutations strategies.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
KyleAMathews and others added 5 commits October 22, 2025 13:15
Make it crystal clear that debounce/throttle only allow one pending tx (collecting mutations) and one persisting tx at a time, while queue guarantees each mutation becomes a separate tx processed in order.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add new "Paced Mutations" section to mutations.md covering:
- Introduction to paced mutations and TanStack Pacer
- Key design differences (debounce/throttle vs queue)
- Detailed examples for each strategy (debounce, throttle, queue)
- Guidance on choosing the right strategy
- React hook usage with usePacedMutations
- Non-React usage with createPacedMutations

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
The id property doesn't make sense for paced mutations because:
- Queue strategy creates separate transactions per mutate() call
- Debounce/throttle create multiple transactions over time
- Users shouldn't control internal transaction IDs

Changed PacedMutationsConfig to explicitly define only the properties
that make sense (mutationFn, strategy, metadata) instead of extending
TransactionConfig.

This prevents TypeScript from accepting invalid configuration like:
  usePacedMutations({ id: 'foo', ... })

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Fixed issue where wrapping usePacedMutations in another hook would
recreate the instance on every render when passing strategy inline:

Before (broken):
  usePacedMutations({ strategy: debounceStrategy({ wait: 3000 }) })
  // Recreates instance every render because strategy object changes

After (fixed):
  // Serializes strategy type + options for stable comparison
  // Only recreates when actual values change

Now uses JSON.stringify to create a stable dependency from the
strategy's type and options, so the instance is only recreated when
the strategy configuration actually changes, not when the object
reference changes.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Add comprehensive tests to verify that usePacedMutations doesn't
recreate the instance unnecessarily when wrapped in custom hooks.

Tests cover:
1. Basic memoization - instance stays same when strategy values are same
2. User's exact scenario - custom hook with inline strategy creation
3. Proper recreation - instance changes when strategy options change

These tests verify the fix for the bug where wrapping usePacedMutations
in a custom hook with inline strategy would recreate the instance on
every render.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Copy link
Collaborator

@samwillis samwillis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this a lot, we're nearly there I think.

My final question is on the queued mutation, the queue is scoped to a single mutation function, but I suspect in reality there will be multiple mutation functions doing different things: create an issue, then create a comment. I may have missed it but does this support that?

Maybe the queue options can take a key, or a queue instance?

I think it could also be worth being more explicit in the docs how the queue version is different than normal mutations - maybe explain in the docs that normal mutations are not queued, then cross linking to the queued version.

@KyleAMathews
Copy link
Collaborator Author

the queue is scoped to a single mutation function, but I suspect in reality there will be multiple mutation functions doing different things: create an issue, then create a comment. I may have missed it but does this support that?

Yeah each hook creates a separate mutator — this is the same as the useMutation hook from Query.

So e.g.

const saveComment = usePacedMutatioin(...)

// in an onChange for the textarea
onChange={(e) => saveComment(e.value)}

@KyleAMathews
Copy link
Collaborator Author

Re docs — this is what I put there — seems clear enough?


Key Design

The fundamental difference between strategies is how they handle transactions:

Debounce/Throttle: Only one pending transaction (collecting mutations) and one persisting transaction (writing to backend) at a time. Multiple rapid mutations automatically merge together into a single transaction.

Queue: Each mutation creates a separate transaction, guaranteed to run in the order they're made (FIFO by default, configurable to LIFO). All mutations are guaranteed to persist.

Wrap the user-provided mutationFn in a stable callback using useRef,
so that even if the mutationFn reference changes on each render,
the paced mutations instance is not recreated.

This fixes the bug where:
1. User types "123" in a textarea
2. Each keystroke recreates the instance (new mutationFn on each render)
3. Each call to mutate() gets a different transaction ID
4. Old transactions with stale data (e.g. "12") are still pending
5. When they complete, they overwrite the correct "123" value

Now the mutationFn identity is stable, so the same paced mutations
instance is reused across renders, and all mutations during the
debounce window batch into the same transaction.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants