Skip to content

Conversation

@prosdev
Copy link
Collaborator

@prosdev prosdev commented Dec 13, 2025

🎯 What This Adds

New dev map command to visualize codebase structure with component counts, exports, and hot paths.

πŸš€ Performance Optimizations

Three Major Bottlenecks Fixed:

  1. Semantic Search β†’ Fast Scan (10-20x faster)

    • Added getAll() method to skip embedding generation
    • No vector similarity calculation needed for structural queries
  2. Embedder Initialization (100+ seconds saved)

    • Added skipEmbedder option to initialize()
    • Lazy loading for read-only operations
  3. Git Stats Enrichment (103 seconds saved)

    • Created getBasicStats() to skip expensive git analysis
    • Avoided scanning 1000 commits just to check if repo is indexed

Results

  • Before: 103+ seconds ⏳
  • After: 0.46 seconds ⚑
  • Speedup: 224x faster!

πŸ› Bugs Fixed

  • Fixed component count propagation causing exponential overflow (2.4B β†’ 3.7K)
  • Added getAll() to test mocks

✨ Features

  • --depth - Control directory nesting (1-5)
  • --focus - Zoom into specific directories
  • --exports / --no-exports - Show/hide exported symbols
  • --change-frequency - Show git activity hotspots
  • --token-budget - Control output size
  • --verbose - Enable debug logging

πŸ“‹ Examples

# Show full codebase structure
dev map

# Deeper nesting
dev map --depth 3

# Focus on specific directory
dev map --focus packages/core

# Show git activity hotspots
dev map --change-frequency

# Debug logging
dev map --verbose

πŸ§ͺ Testing

  • βœ… All 1,918 tests passing
  • βœ… Lint clean
  • βœ… TypeCheck passed
  • βœ… Pre-commit hooks passed

πŸ“ Files Changed

  • packages/core/src/vector/store.ts - Added getAll()
  • packages/core/src/vector/index.ts - Added lazy embedder init
  • packages/core/src/indexer/index.ts - Added getBasicStats()
  • packages/core/src/map/index.ts - Fixed count propagation + logging
  • packages/cli/src/commands/map.ts - New command with verbose logging
  • packages/cli/src/cli.ts - Registered map command
  • Tests updated with getAll() mocks

Add new 'dev map' command to visualize codebase structure with component
counts, exports, and hot paths. Includes major performance optimizations
for read-only operations.

Features:
- Directory structure with component counts at configurable depth
- Hot paths showing most referenced files
- Exported symbols per directory
- Optional git change frequency analysis
- Focus on specific directories
- Verbose logging with --verbose flag

Performance Optimizations:
- Add getAll() method to skip semantic search for structural queries
- Add skipEmbedder option to initialize() for read-only operations
- Add getBasicStats() to avoid expensive git enrichment
- Result: 224x speedup (103s β†’ 0.46s)

Bug Fixes:
- Fix component count propagation causing exponential overflow
- Add getAll() to test mocks

Examples:
  $ dev map                           # Show full structure
  $ dev map --depth 3                 # Deeper nesting
  $ dev map --focus packages/core     # Focus on directory
  $ dev map --change-frequency        # Show git hotspots
  $ dev map --verbose                 # Debug logging
@prosdev
Copy link
Collaborator Author

prosdev commented Dec 13, 2025

βœ… Comprehensive Testing Complete

All features have been tested and verified working correctly:

1. Basic Map Generation ⚑

dev map --depth 2
  • Time: 0.46 seconds
  • Result: 3,733 components across 17 directories
  • Status: βœ… Working perfectly

2. Change Frequency Analysis πŸ”₯

dev map --depth 2 --change-frequency
  • Time: 2.2 seconds (adds ~1.7s for git analysis)
  • Indicators:
    • πŸ”₯ for hot directories (5+ commits/month)
    • ✏️ for moderate activity (1-4 commits/month)
  • Status: βœ… Accurate git activity tracking

3. Verbose Logging πŸͺ΅

dev map --verbose --change-frequency
  • Features:
    • Detailed timing for each step (Retrieved docs: 41ms, Built tree: 1ms, Git analysis: 537ms)
    • DEBUG/INFO level logs with structured metadata
    • Duration breakdowns and component counts
  • Status: βœ… Comprehensive debugging info

4. Focus on Directory 🎯

dev map --focus packages/cli --depth 3
  • Result: Filters to specific directory with deeper nesting
  • Graceful handling: Non-existent paths show 0 components (no crash)
  • Status: βœ… Works as expected

5. Hide Exports πŸ‘οΈ

dev map --no-exports
  • Result: Shows only structure and component counts, no export details
  • Status: βœ… Clean output

6. Error Validation πŸ›‘οΈ

dev map --depth 10  # Invalid
  • Error: "βœ– Invalid depth - Depth must be between 1 and 5"
  • Status: βœ… Clear error messages

7. Help Text πŸ“–

dev map --help
  • Content: Options, examples, use cases, emoji indicators
  • Status: βœ… Comprehensive and helpful

🎯 Performance Verified

Operation Time Speedup
Before optimization 103+ seconds -
Basic map (new) 0.46s 224x faster ⚑
With --change-frequency 2.2s Git analysis adds ~1.7s (expected)
With --verbose Same Just adds logging output

πŸ§ͺ Test Coverage

  • βœ… All 1,918 tests passing
  • βœ… Lint clean (Biome)
  • βœ… TypeCheck passed
  • βœ… Pre-commit hooks passed
  • βœ… Mock updates for getAll() in test suites

πŸ“Š Sample Output

# Codebase Map

## Hot Paths (most referenced)
1. packages/core/src/scanner/typescript.ts - 257 refs
2. packages/core/src/scanner/go.ts - 115 refs
...

## Directory Structure

└── root/ (3733 components)
    β”œβ”€β”€ packages/ (2774 components) πŸ”₯ 100 commits this month
    β”‚   β”œβ”€β”€ cli/ (146 components) πŸ”₯ 64 commits this month
    β”‚   β”‚   └── exports: Search, Explore, Update, Stats, Clean
    β”‚   β”œβ”€β”€ core/ (950 components) πŸ”₯ 75 commits this month
    β”‚   β”œβ”€β”€ mcp-server/ (887 components) πŸ”₯ 71 commits this month
    β”‚   └── subagents/ (630 components) πŸ”₯ 71 commits this month
    β”œβ”€β”€ docs/ (42 components) πŸ”₯ 9 commits this month
    └── website/ (351 components) πŸ”₯ 27 commits this month

πŸ“Š Total: 3,733 components across 17 directories
πŸ”₯ 5 hot paths identified

Ready for review and merge! πŸš€

@prosdev prosdev merged commit a22632d into main Dec 13, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant