feat(eval): add eval id to metrics #995
Conversation
- pass down eval id to be added to model inference Details Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Summary of ChangesHello @karthikrab, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request implements a new feature to associate model inference metrics with a specific evaluation ID. By passing down an Highlights
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request successfully adds an evaluation_id to the metrics pipeline, allowing for better tracking of evaluation-related requests. The changes are well-propagated through the Python services, the Rust gateway, and into the database schema and migrations. The logic for adding the evaluation_id to API key metadata, request headers, and analytics records is sound. I've included one suggestion to refactor a function in the analytics middleware to improve code clarity and maintainability by reducing repetition.
| fn extract_auth_metadata(headers: &HeaderMap, record: &mut GatewayAnalyticsDatabaseInsert) { | ||
| // Extract api_key_id | ||
| if let Some(api_key_id) = headers.get("x-tensorzero-api-key-id") { | ||
| if let Ok(id_str) = api_key_id.to_str() { | ||
| record.api_key_id = Some(id_str.to_string()); | ||
| tracing::debug!("Captured api_key_id for analytics: {}", id_str); | ||
| } | ||
| } | ||
|
|
||
| // Extract user_id | ||
| if let Some(user_id) = headers.get("x-tensorzero-user-id") { | ||
| if let Ok(id_str) = user_id.to_str() { | ||
| record.user_id = Some(id_str.to_string()); | ||
| tracing::debug!("Captured user_id for analytics: {}", id_str); | ||
| } | ||
| } | ||
|
|
||
| // Extract project_id | ||
| if let Some(project_id) = headers.get("x-tensorzero-project-id") { | ||
| if let Ok(id_str) = project_id.to_str() { | ||
| if let Ok(uuid) = Uuid::parse_str(id_str) { | ||
| record.project_id = Some(uuid); | ||
| tracing::debug!("Captured project_id for analytics: {}", id_str); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Extract endpoint_id | ||
| if let Some(endpoint_id) = headers.get("x-tensorzero-endpoint-id") { | ||
| if let Ok(id_str) = endpoint_id.to_str() { | ||
| if let Ok(uuid) = Uuid::parse_str(id_str) { | ||
| record.endpoint_id = Some(uuid); | ||
| tracing::debug!("Captured endpoint_id for analytics: {}", id_str); | ||
| } | ||
| } | ||
| } | ||
|
|
||
| // Extract evaluation_id | ||
| if let Some(evaluation_id) = headers.get("x-tensorzero-evaluation-id") { | ||
| if let Ok(id_str) = evaluation_id.to_str() { | ||
| if let Ok(uuid) = Uuid::parse_str(id_str) { | ||
| record.evaluation_id = Some(uuid); | ||
| tracing::debug!("Captured evaluation_id for analytics: {}", id_str); | ||
| } | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
This function contains a lot of repetitive code for extracting different metadata fields from the headers. This can be refactored to be more concise and maintainable by using and_then to chain the operations, which is more idiomatic in Rust and reduces nesting. This will make the function shorter and easier to read.
/// Extract auth metadata from request headers (injected by auth middleware)
fn extract_auth_metadata(headers: &HeaderMap, record: &mut GatewayAnalyticsDatabaseInsert) {
if let Some(api_key_id) = headers.get("x-tensorzero-api-key-id").and_then(|v| v.to_str().ok()) {
record.api_key_id = Some(api_key_id.to_string());
tracing::debug!("Captured api_key_id for analytics: {}", api_key_id);
}
if let Some(user_id) = headers.get("x-tensorzero-user-id").and_then(|v| v.to_str().ok()) {
record.user_id = Some(user_id.to_string());
tracing::debug!("Captured user_id for analytics: {}", user_id);
}
if let Some(uuid) = headers.get("x-tensorzero-project-id").and_then(|v| v.to_str().ok()).and_then(|s| Uuid::parse_str(s).ok()) {
record.project_id = Some(uuid);
tracing::debug!("Captured project_id for analytics: {}", uuid);
}
if let Some(uuid) = headers.get("x-tensorzero-endpoint-id").and_then(|v| v.to_str().ok()).and_then(|s| Uuid::parse_str(s).ok()) {
record.endpoint_id = Some(uuid);
tracing::debug!("Captured endpoint_id for analytics: {}", uuid);
}
if let Some(uuid) = headers.get("x-tensorzero-evaluation-id").and_then(|v| v.to_str().ok()).and_then(|s| Uuid::parse_str(s).ok()) {
record.evaluation_id = Some(uuid);
tracing::debug!("Captured evaluation_id for analytics: {}", uuid);
}
}
- pass down eval id to be added to model inference Details Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
- pass down eval id to be added to model inference Details Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
Generated with Claude Code
Co-Authored-By: Claude noreply@anthropic.com