Database Indexing Optimization (Issue #1353)

[rakdutta]

Closes #1353, #1663
This PR implements comprehensive database indexing optimization to improve query performance across the entire MCPGateway application. The migration adds strategic indexes for foreign keys and common query patterns.

What's Changed

Phase 0: Index Naming Standardization

• Renames all existing ix_ prefixed indexes to idx_ for consistency
• Handles database-specific rename operations (PostgreSQL, MySQL, SQLite)

Phase 0.5: Duplicate Index Cleanup

• Detects and removes duplicate indexes where both ix_ and idx_ versions exist
• Prevents index duplication and saves storage space

observability_events|severity|2|idx_observability_events_severity,ix_observability_events_severity
observability_events|span_id|2|idx_observability_events_span_id,ix_observability_events_span_id
observability_events|timestamp|2|idx_observability_events_timestamp,ix_observability_events_timestamp
observability_metrics|resource_type|2|idx_observability_metrics_resource_type,ix_observability_metrics_resource_type
observability_metrics|trace_id|2|idx_observability_metrics_trace_id,ix_observability_metrics_trace_id
observability_saved_queries|user_email|2|idx_observability_saved_queries_user_email,ix_observability_saved_queries_user_email
observability_spans|parent_span_id|2|idx_observability_spans_parent_span_id,ix_observability_spans_parent_span_id
observability_spans|resource_name|2|idx_observability_spans_resource_name,ix_observability_spans_resource_name
observability_spans|resource_type|2|idx_observability_spans_resource_type,ix_observability_spans_resource_type
observability_spans|start_time|2|idx_observability_spans_start_time,ix_observability_spans_start_time
observability_spans|trace_id|2|idx_observability_spans_trace_id,ix_observability_spans_trace_id
observability_traces|start_time|2|idx_observability_traces_start_time,ix_observability_traces_start_time
observability_traces|user_email|2|idx_observability_traces_user_email,ix_observability_traces_user_email
performance_snapshots|timestamp|2|idx_performance_snapshots_timestamp,ix_performance_snapshots_timestamp

Phase 1: Foreign Key Indexes (50+ indexes)

Adds indexes for foreign keys that were missing them, improving:

• JOIN performance - Eliminates full table scans during joins
• Constraint checks - Faster INSERT/UPDATE/DELETE operations
• Cascading operations - Efficient cascading deletes/updates

Key areas covered:

• Role and RBAC relationships
• Team management (members, invitations, join requests)
• Core entities (tools, resources, prompts, servers, gateways)
• Metrics tracking
• OAuth and authentication tokens
• SSO sessions

Phase 2: Composite Indexes (30+ indexes)

Adds multi-column indexes for frequently used query patterns:

Team Management

• User + team + active status lookups
• Team visibility + active + timestamp filtering
• Invitation and join request status queries

Core Entities (Tools/Resources/Prompts/Servers)

• Team + visibility + enabled + created_at (team-scoped listings)
• Visibility + enabled + created_at (public listings)

Observability

• User activity tracking (user + status + time)
• Error analysis (status + method + time)
• Resource monitoring (resource_type + status + time)

Authentication & Tokens

• User/team token listings (user/team + active + created)
• Auth event auditing (user + event_type + timestamp)
• Session lookups (provider + user + created)

Metrics

• Time-series queries (entity_id + timestamp)

RBAC

• Permission checks (user + scope + active)
• Role membership queries (role + scope + active)

Phase 3 - Foreign Key Constraint Fixes:

closes #1663
Adds ON DELETE CASCADE to email_team_member_history.team_member_id foreign key
to fix PostgreSQL constraint violations when deleting users (Issue: user deletion fails).

Performance Impact

• Faster queries: Eliminates full table scans for common query patterns
• Reduced I/O: Indexes allow direct data access without scanning entire tables
• Better scalability: Performance improvements scale with data growth
• Optimized joins: Foreign key indexes dramatically improve multi-table queries

Migration Safety

• Uses safe helper functions that check for existing indexes before creation
• Handles database-specific operations (PostgreSQL, MySQL, SQLite)
• Provides detailed logging during migration
• Includes complete downgrade path for rollback

Testing

• Migration tested on SQLite, PostgreSQL, and MySQL
• Verified index creation and duplicate detection
• Confirmed no breaking changes to existing functionality