feat(legal): add citation_exists requirement and real-data tests for case validation #191
Conversation
…case validation Implements a new requirement that checks if a case name exists in the cases_metadata database using actual JSON files. Adds parameterized pytest tests that validate case detection against real data.
Merge ProtectionsYour pull request matches the following merge protections and will not be merged until they are valid. 🟢 Enforce conventional commitWonderful, this rule succeeded.Make sure that we follow https://www.conventionalcommits.org/en/v1.0.0/
|
There was a problem hiding this comment.
The implementation of the requirement should be in mellea/stdlib/reqlib/legal.py.
| # create mock context | ||
| ctx = MockContext(case_name) | ||
| # path to metadata folder | ||
| db_folder = "/Users/anooshkapendyal/Desktop/mellea/mellea/test/stdlib_basics/legal/cases_metadata" |
|
|
||
| return name.strip() | ||
|
|
||
| def citation_exists(ctx: Context, folder_path: str) -> ValidationResult: |
There was a problem hiding this comment.
We should always operate over strings.
So here, instead of folder_path, pass in a list of the actual contents of that folder, whatever those are. In this case, the metadata json files.
If these were inteded as LLM inputs, then you'd want to pass them in as CBlocks rather than strs.
The type signature of this function should probably be:
def citation_exists(ctx: Context, folder_path: list[dict]) -> ValidationResult: ...There was a problem hiding this comment.
If you find that the case metadata is too big to pass in directly, then you could instead use Generator[...] instead of list[...]
| """ | ||
| Checks if the output case name exists in the provided case metadata database. | ||
| """ | ||
| def __init__(self, folder_path: str): |
There was a problem hiding this comment.
Should be case_metadata instead of folder_path.
2 participants
Implements a new requirement that checks if a case name exists in the cases_metadata database using actual JSON files. Adds parameterized pytest tests that validate case detection against real data.