feat: fallbacks added to vk provider routing

[Pratham-Mishra04]

Summary

Briefly explain the purpose of this PR and the problem it solves.

Changes

• What was changed and why
• Any notable design decisions or trade-offs

Type of change

• Bug fix
• Feature
• Refactor
• Documentation
• Chore/CI

Affected areas

• Core (Go)
• Transports (HTTP)
• Providers/Integrations
• Plugins
• UI (Next.js)
• Docs

How to test

Describe the steps to validate this change. Include commands and expected outcomes.

# Core/Transports
go version
go test ./...

# UI
cd ui
pnpm i || npm i
pnpm test || npm test
pnpm build || npm run build

If adding new configs or environment variables, document them here.

Screenshots/Recordings

If UI changes, add before/after screenshots or short clips.

Breaking changes

• Yes
• No

If yes, describe impact and migration instructions.

Related issues

Link related issues and discussions. Example: Closes #123

Security considerations

Note any security implications (auth, secrets, PII, sandboxing, etc.).

Checklist

• I read docs/contributing/README.md and followed the guidelines
• I added/updated tests where appropriate
• I updated documentation where needed
• I verified builds succeed (Go and UI)
• I verified the CI pipeline passes locally if applicable

[Pratham-Mishra04]

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

• feat: fallbacks added to vk provider routing #557 👈 (View in Graphite)
• v1.3.0 #507 : 4 other dependent PRs (#470 , #536 , #556 and 1 other)
• main

This stack of pull requests is managed by Graphite. Learn more about stacking.

[coderabbitai]

Caution

Review failed

The pull request is closed.

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • SGL provider now supports embeddings.
  • Automatic weighted provider selection with generated fallbacks.
  • Improved Gemini handling: correctly distinguishes transcriptions from chat completions.

• Improvements

  • Logging now links parent/child requests; plugin name shortened to “logging”.
  • Default config directory now uses the OS-specific location.

• UI

  • Not Found page now links to Logs.
  • Virtual Keys and Teams/Customers pages use governance-aware loading with clearer consolidated errors and manual refresh.

• Configuration

  • Added settings for allowed origins and allowing direct keys.
  • Provider configs are preloaded when fetching virtual keys.

Walkthrough

Adds fallback request ID propagation for fallbacks, updates logging to capture parent-child request IDs, implements weighted provider routing with auto-generated fallbacks, enables SGL embeddings and Gemini transcription-aware responses, preloads provider configs in RDB, adds new configstore migrations, adjusts HTTP server default config dir, and updates UI pages to lazy-load governance-gated data.

Changes

Cohort / File(s)	Summary
Fallback request ID propagation core/bifrost.go, core/schemas/bifrost.go, plugins/logging/main.go, plugins/logging/operations.go, framework/logstore/tables.go	Generate a UUID per fallback, attach via new context key; logging plugin uses fallback ID as RequestID and stores original as ParentRequestID; log table gains ParentRequestID; insert operation updated to accept parentRequestID.
Dependencies core/go.mod	Adds direct dependency github.com/google/uuid v1.6.0; removes it from indirect block.
SGL embeddings enablement core/providers/sgl.go, tests/core-providers/sgl_test.go	Implements real embeddings call via shared OpenAI-style helper; tests updated with explicit models and enable text completion and embeddings.
Gemini transcription handling core/schemas/providers/gemini/chat.go	Distinguishes transcription vs chat completion; constructs appropriate response objects, including choices, usage, and finish reasons.
Provider concurrency init cleanup core/bifrost.go	Consolidates wait group retrieval and Add calls using a cached reference; logs for fallback success downgraded to Debug.
Configstore migrations framework/configstore/migrations.go	Adds migrations for allowed_origins_json and allow_direct_keys columns; integrates into migration sequence.
RDB eager loading framework/configstore/rdb.go	Preloads ProviderConfigs in GetVirtualKeys and GetVirtualKey.
VK routing + fallbacks transports/bifrost-http/handlers/middlewares.go, transports/bifrost-http/handlers/completions.go	Implements weighted random provider selection; auto-generates sorted fallback providers when absent; parseFallbacks now filters invalid entries via append.
HTTP server config dir transports/bifrost-http/handlers/server.go	Default app dir set to empty to use OS config dir; logs resolved configDir during bootstrap.
UI governance-gated lazy loading ui/app/virtual-keys/page.tsx, ui/app/teams-customers/page.tsx	Switches to lazy RTK queries; loads core config first; conditionally fetches governance data; updates loading/error handling and refresh logic.
UI NotFound tweaks ui/app/not-found.tsx	Adjusts layout sizing and changes “Go home” link to “/logs”.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  actor Client
  participant HTTP as HTTP Handler
  participant Router as VKProviderRoutingMiddleware
  participant Downstream as Bifrost Core
  Note over Router: Weighted selection + auto-fallbacks
  Client->>HTTP: POST /v1/chat/completions (with VK)
  HTTP->>Router: Middleware
  Router->>Router: Select provider by weight
  Router->>Router: Build fallbacks (exclude chosen, sort by weight)
  Router->>HTTP: Set model and fallbacks in request body
  HTTP->>Downstream: Forward request (model + fallbacks)
  Downstream-->>Client: Response/stream

Loading

sequenceDiagram
  autonumber
  actor Client
  participant Core as Core (bifrost.go)
  participant Provider as Provider Worker
  participant Log as Logging Plugin
  Note over Core,Provider: Fallback handling with request IDs
  Client->>Core: Request
  Core->>Provider: Try primary
  alt failure -> fallback
    Core->>Core: Generate fallback UUID
    Core->>Provider: Invoke fallback with ctx[fallback-request-id]
    Core->>Log: PreHook (RequestID=fallback, ParentRequestID=original)
    Provider-->>Core: Result
    Core->>Log: PostHook (uses fallback RequestID)
  else success
    Core->>Log: PreHook (RequestID=original)
    Provider-->>Core: Result
    Core->>Log: PostHook
  end
  Core-->>Client: Final response

Loading

sequenceDiagram
  autonumber
  actor Client
  participant SGL as SGL Provider
  participant Core as SGL Adapter
  Note over Core: Embeddings implemented
  Client->>Core: Embeddings request
  Core->>SGL: POST /v1/embeddings
  SGL-->>Core: Embeddings response
  Core-->>Client: Normalized response

Loading

sequenceDiagram
  autonumber
  actor Client
  participant Gemini as Gemini Adapter
  Note over Gemini: Transcription-aware
  Client->>Gemini: Chat response
  Gemini->>Gemini: Detect transcription vs chat
  alt Transcription
    Gemini-->>Client: audio.transcription response
  else Chat completion
    Gemini-->>Client: chat.completion with choices and usage
  end

Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

Thump-thump! I log with glee,
New IDs hop from tree to tree.
Providers weighed, fallbacks queue—
SGL embeds? Now they do!
Gemini sings, transcribes the breeze,
UI waits for govern-keys.
Carrot-shaped commits, ship with ease. 🥕🐇

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch 10-06-feat_fallbacks_added_to_vk_provider_routing

---

📜 Recent review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 48616ed and aeef5b5.

📒 Files selected for processing (17)

• core/bifrost.go (7 hunks)
• core/go.mod (1 hunks)
• core/providers/sgl.go (1 hunks)
• core/schemas/bifrost.go (1 hunks)
• core/schemas/providers/gemini/chat.go (2 hunks)
• framework/configstore/migrations.go (2 hunks)
• framework/configstore/rdb.go (2 hunks)
• framework/logstore/tables.go (1 hunks)
• plugins/logging/main.go (6 hunks)
• plugins/logging/operations.go (2 hunks)
• tests/core-providers/sgl_test.go (2 hunks)
• transports/bifrost-http/handlers/completions.go (1 hunks)
• transports/bifrost-http/handlers/middlewares.go (3 hunks)
• transports/bifrost-http/handlers/server.go (2 hunks)
• ui/app/not-found.tsx (1 hunks)
• ui/app/teams-customers/page.tsx (2 hunks)
• ui/app/virtual-keys/page.tsx (1 hunks)

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}