Ability to create collections with parameterized query functions

[medreseli]

After reading Tanstack DB documentation, I was not able to find an example that fetches data from server with parameters. All examples show how to fetch all the data. I need to use project ID to fetch that project's data, not everything else.

Fow example, this creates a collection. queryFn: async () has no parameter.

const todoCollection = createCollection(
  queryCollectionOptions({
    queryKey: ['todos'],
    queryFn: async () => {
      const response = await fetch('/api/todos')
      return response.json()
    },
    getKey: (item) => item.id,
    // Handle all CRUD operations
    onInsert: async ({ transaction }) => {
      const { modified: newTodo } = transaction.mutations[0]
      await fetch('/api/todos', {
        method: 'POST',
        body: JSON.stringify(newTodo),
      })
    },
    onUpdate: async ({ transaction }) => {
      const { original, modified } = transaction.mutations[0]
      await fetch(`/api/todos/${original.id}`, {
        method: 'PUT', 
        body: JSON.stringify(modified),
      })
    },
    onDelete: async ({ transaction }) => {
      const { original } = transaction.mutations[0]
      await fetch(`/api/todos/${original.id}`, { method: 'DELETE' })
    },
  })
)

Since, todoCollection is being used in a component like below, I can not put it in a useEffect to either.

function TodoList() {
  // Basic filtering and sorting
  const { data: incompleteTodos } = useLiveQuery((q) =>
    q.from({ todo: todoCollection })
     .where(({ todo }) => eq(todo.completed, false))
     .orderBy(({ todo }) => todo.createdAt, 'desc')
  )

  // Transform the data
  const { data: todoSummary } = useLiveQuery((q) =>
    q.from({ todo: todoCollection })
     .select(({ todo }) => ({
       id: todo.id,
       summary: `${todo.text} (${todo.completed ? 'done' : 'pending'})`,
       priority: todo.priority || 'normal'
     }))
  )

  return <div>{/* Render todos */}</div>
}

What I want is something like below:

const todoCollection = createCollection(
  queryCollectionOptions({
    queryKey: ['todos'],
    queryFn: async (id: string) => {
      const response = await fetch('/api/todos/' + id);
      return response.json();
    },
    getKey: item => item.id,
  })
);

Is there a way to do this?

[giladv]

exactly what i'm not getting about this library. are we supposed to load everything?
what happens if theres a /ORG_ID/todos ?

[KyleAMathews]

Collections are just Maps that sync basically — you can create as many of them as you want in whatever fashion you want.

I've been meaning to document a collection factory pattern which is what it sounds like you two want. I'll do this soon. This is what we see a lot of people doing something like this:

const cache = new Map<string, ReturnType<typeof createCollection>>();

const getTodoCollection = (id: string) => {
  if (!cache.has(id)) {
    const collection = createCollection(
      queryCollectionOptions({
        queryKey: ['todos', id],
        queryFn: async () => {
          const response = await fetch('/api/todos/' + id);
          return response.json();
        },
        getKey: item => item.id,
      })
    );
    
    collection.on('status:change', ({ status }) => {
      if (status === 'cleaned-up') {
        cache.delete(id);
      }
    });
    
    cache.set(id, collection);
  }
  return cache.get(id)!;
};

We're also working on "query-driven sync" which will let a collection load data on demand based on what live queries you write — #612 — we'll have more on this soon.

[litzebauer]

@KyleAMathews I am following your collection factory pattern but the collection never fires 'status:change' with the status 'cleaned-up'. It looks like on clean up the event listeners are cleared before the status is changed to 'cleaned-up'. https://github.com/TanStack/db/blob/main/packages/db/src/collection/lifecycle.ts#L254-L269

[KyleAMathews]

@litzebauer oh heh — that's less than useful. Here's a PR w/ a fix #714

[litzebauer]

Thanks a lot! Looks good! Wasn't sure if I should submit a PR or call issue out first.