[Bug]: trajectory_exact_match fails with "Required field is not set" in evaluating_adk_agent.ipynb #2417
Description
File Name
gemini/evaluation/evaluating_adk_agent.ipynb
What happened?
Description
When running the trajectory evaluation section (cell 43), the following error occurs:
Error:
Error: 400 List of found errors:
Field: trajectory_exact_match_input.instances[0].reference_trajectory
Message: Required field is not set.
Expected Behavior
Trajectory metrics (trajectory_exact_match, trajectory_in_order_match,
trajectory_any_order_match, trajectory_precision, trajectory_recall) should execute
successfully.
Root Cause
Cell 28 creates the evaluation dataset with "predicted_trajectory" as the column name, but
Vertex AI Gen AI Evaluation service requires "reference_trajectory" for ground truth
comparisons.
Current code in cell 28:
eval_data = {
"prompt": [...],
"predicted_trajectory": [...] # Wrong column name
}
Should be:
eval_data = {
"prompt": [...],
"reference_trajectory": [...] # Correct column name
}
Steps to Reproduce
1. Open the notebook in Colab or Vertex AI Workbench
2. Run cells 1-42 successfully
3. Run cell 43 (trajectory evaluation)
4. Observe the 400 error
Impact
This bug prevents users from running trajectory evaluation metrics, which is a core feature of
the ADK agent evaluation tutorial.
### Relevant log output
```shell
Error encountered for metric trajectory_exact_match at dataset index 1: Error: 400 List of found
errors:1.Field: trajectory_exact_match_input.instances[0].reference_trajectory;
Message: Required field is not set.[field_violations {
field: "trajectory_exact_match_input.instances[0].reference_trajectory"
description: "Required field is not set."Code of Conduct
- I agree to follow this project's Code of Conduct
Metadata
Metadata
Assignees
Labels
No labels