Fixes to llm output parsing when using LLM based ranking

[amrit110]

This pull request introduces improvements to schema item filtering, LLM output sanitization, and prompt clarity for schema ranking. The most important changes are the addition of robust parsing and validation utilities for schema items, enhanced pre-processing and extraction of LLM-generated JSON, and improved handling of invalid LLM outputs. The schema ranking prompt has also been rewritten for clarity and strict output requirements.

Schema item parsing and filtering improvements:

• Added utility functions (_parse_schema_item, _parse_column_ref, _get_foreign_key) to robustly parse and validate schema item references, and to extract foreign key relationships in src/pipe/add_schema.py. This ensures only valid columns and their foreign keys are included during schema filtering.
• Refactored the filter_schema function to use these utilities, improving the reliability of schema filtering and handling edge cases.

LLM output sanitization and extraction:

• Introduced _preprocess_json_string in src/pipe/llm_util.py to fix common LLM formatting errors in JSON output, such as array termination, missing quotes, and empty items, before parsing.
• Updated extract_json to pre-process LLM output before parsing, improving robustness against malformed JSON.
• Changed logging in extract_object to debug level for failed extractions, reducing noise in production logs.

Schema ranking and prompt handling:

• Added _sanitize_schema_item and improved _process_output in src/pipe/rank_schema_llm.py to validate and sanitize LLM outputs, fallback to all schema items if output is invalid, and log warnings when necessary.
• Rewrote the schema ranking prompt in src/pipe/rank_schema_prompts/v1.py to clarify input/output formats, requirements, and examples, ensuring LLMs return strictly valid and relevant schema items.