Skip to content

Conversation

@ammesonb
Copy link
Contributor

This PR implements filtering by tags and metadata in the HSG query system, allowing users to narrow down memory retrieval results based on associated tags and metadata
properties without reducing the result count.

Changes Made

1. Type Definitions (src/types/index.ts)

  • Added optional metadata object to query request filters
  • Created reusable MemoryFilters type for type safety across the codebase

2. Core Query Logic (src/hsg/index.ts)

  • Added passesFilters() helper function:
    • Centralized function that checks if a memory passes all applied filters (salience, tags, metadata)
    • Maintains existing salience check
    • Tag/metadata filtering is only applied if non-empty definitions are provided
    • Tag filtering: Returns memories that have at least one matching tag from the provided array
    • Metadata filtering: Returns memories where all specified metadata key-value pairs match exactly
    • Includes error handling for JSON parsing failures
  • Updated hsgQuery() function signature: Changed from individual filter parameters to accept a single filters object for cleaner API
  • Optimized filter application point: Filters are now applied AFTER vector similarity search and waypoint expansion but BEFORE sorting and top-K selection

Why This Approach:

Applying filters after vector search but before top-K selection ensures that:

  1. Vector similarity search finds the most semantically relevant memories
  2. Filters remove irrelevant results
  3. Top K selection returns the desired number of filtered, relevant results

3. API Endpoint (src/server/index.ts)

  • Updated /memory/query POST endpoint to pass the full filters object to hsgQuery()
  • Full backwards compatibility for filters object
  • Maintained sector parameter transformations

4. Tests (tests/backend/api.test.js)

  • Tag filtering tests: Verify only memories with matching tags are returned
  • Metadata filtering tests: Verify only memories with matching metadata properties are returned

5. Manual Testing

  1. Added sample memories:
image
  1. Various queries with different tag/metadata combinations
image
  1. Check salience filters
image
  1. Check empty/null cases for tags and metadata
image

Example Usage

Tag filtering:

POST /memory/query
{
  "query": "machine learning concepts",
  "k": 10,
  "filters": {
    "tags": ["AI", "tutorial"]
  }
}

Metadata filtering:

POST /memory/query
{
  "query": "project documentation",
  "k": 10,
  "filters": {
    "metadata": {
      "source": "github",
      "project": "OpenMemory"
    }
  }
}

Combined filtering:

POST /memory/query
{
  "query": "architecture decisions",
  "k": 10,
  "filters": {
    "tags": ["architecture"],
    "metadata": { "reviewed": true },
    "min_score": 0.5
  }
}

Also updates Makefile for new test filenames, using targets instead of
redefining test steps
@nullure
Copy link
Member

nullure commented Oct 25, 2025

LGTM

@nullure nullure merged commit 044e66d into CaviraOSS:main Oct 25, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants