Skip to content

(EAI-1190) fix eval experiments #845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Aug 4, 2025
Merged

(EAI-1190) fix eval experiments #845

merged 6 commits into from
Aug 4, 2025

Conversation

yakubova92
Copy link
Collaborator

Jira: https://jira.mongodb.org/browse/EAI-1190

Changes

  • using makeGenerateResponse instead of generateResponse in evals/experiments files
  • tsconfig file was excluding "**/*.eval.ts", "**/eval/**/*", I removed this so that these files will be included in the build and the build will fail if there are changes to the config that need to be reflected in evals/experiments files
  • found a build error when doing this (Module '"mongodb-rag-core"' has no exported member 'validateTags'.), which I fixed with the change to the mongodb-rag-core/src/index.ts file

Notes

@yakubova92 yakubova92 requested a review from mongodben July 24, 2025 21:51
Copy link
Collaborator

@mongodben mongodben Aug 4, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't need the .eval.ts files in the actual build output, so i dont think this is the correct approach. rather we could make sure that the evals are being compiled prior to running the main build.

rather, we can just check that the build works using tsconfig.json, which checks all files (whereas tsconfig.build.json only compiles the stuff in the final build).

can update package.json as follows:

{
//other stuff
"scripts": {
    // checks that compiles w/o emitting the compiled files
    "build:check": "tsc --noEmit --project tsconfig.json",
    // automatically runs build:check before build (https://docs.npmjs.com/cli/v8/using-npm/scripts#pre--post-scripts)
    "prebuild": "npm run build:check",
    "build": "rm -rf ./build/ && tsc -b tsconfig.build.json",
//...
},
//...
}

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the suggestion, using the prebuild is a much cleaner way to get this done

Copy link
Collaborator

@mongodben mongodben left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added suggested improvement to the build process. tested on my end and seems to work. pls apply change and validate in CI before merge

@yakubova92 yakubova92 merged commit 8faa7a4 into main Aug 4, 2025
2 checks passed
@yakubova92 yakubova92 deleted the EAI-1190 branch August 4, 2025 20:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants