Feat/budpromt init

[vsraccubits]

No description provided.

[gemini-code-assist]

Summary of Changes

Hello @vsraccubits, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request introduces the initial setup of the budprompt microservice, a core component designed for the comprehensive management of LLM prompts. The service is built using FastAPI, PostgreSQL, Redis, and leverages Dapr for distributed application runtime capabilities. It provides functionalities for dynamic prompt execution, supporting both structured and unstructured inputs and outputs, and integrates advanced validation mechanisms that can be defined using natural language prompts. Furthermore, it includes robust streaming capabilities for real-time LLM responses and is designed to integrate seamlessly with a BudServe inference gateway. The PR also establishes a complete development and deployment infrastructure, alongside a thorough test suite.

Highlights

• Initial budprompt Microservice Setup: This PR introduces the foundational structure for the budprompt service, including its core application logic, configuration, and infrastructure setup, laying the groundwork for LLM prompt management.
• LLM Prompt Execution Engine: A robust engine for executing LLM prompts has been implemented, supporting both structured and unstructured inputs and outputs, and designed for seamless integration with a BudServe inference gateway.
• Dynamic Input/Output Validation: Advanced validation capabilities for LLM inputs and outputs are introduced, allowing validation rules to be defined using natural language prompts, complemented by built-in retry mechanisms for enhanced reliability.
• Streaming API Support: The service now supports Server-Sent Events (SSE) for real-time streaming of LLM responses, incorporating incremental validation during the streaming process to ensure data integrity.
• Dapr Integration: Dapr components, including state store, pub/sub, secret store, and config store, are configured to handle distributed application concerns, ensuring scalability, resilience, and simplified microservice development.
• Comprehensive Testing: A comprehensive suite of integration tests has been added, covering health checks, input/output validation, streaming, various model parameters, message roles, and Jinja2 template rendering, ensuring the service's robustness.
• Automated Development Workflow: The development workflow is streamlined with the setup of pre-commit hooks, commitlint, and detailed development/deployment scripts (Docker Compose, Helm charts), promoting code quality and efficient development cycles.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point in your pull request via creating an issue comment (i.e. comment on the pull request page) using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in issue comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[gemini-code-assist]

Code Review

This pull request initializes the budprompt service with a comprehensive set of features, including core application logic, Dapr integration, deployment configurations, and extensive tests. The overall structure is well-organized. However, there are several critical security vulnerabilities related to the use of exec with user input and improper handling of secrets in deployment configurations. Additionally, there are high-severity issues in database configuration and deployment scripts that will likely cause runtime failures. Several medium-severity issues related to code style, error handling, and configuration consistency should also be addressed to improve maintainability and robustness.

[gemini-code-assist]

The use of exec() on validator_code, which is generated from a user-provided validation_prompt, introduces a critical remote code execution (RCE) vulnerability. A malicious user could craft a validation_prompt that generates and executes arbitrary Python code on the server. The feature to generate validators from natural language is powerful, but it must be implemented safely. Instead of exec, consider using a safer alternative like a restricted execution environment or parsing the validation logic into an AST and only allowing a safe subset of Python's features.

[gemini-code-assist]

Similar to the issue in streaming_validation.py, the use of exec() on validator_code generated from user input (validation_prompt) creates a critical remote code execution (RCE) vulnerability. An attacker could provide a malicious prompt to execute arbitrary code on your server. This is a significant security risk and should be addressed immediately. Please consider using a sandboxed environment or a custom DSL parser instead of exec.

[gemini-code-assist]

There is a missing comma between the strings in the classifiers list. This will cause a SyntaxError when setup.py is executed.

Suggested change

	"Framework :: Pytest" "Programming Language :: Python :: 3",
	"Framework :: Pytest", "Programming Language :: Python :: 3",

[gemini-code-assist]

The environment variables for PostgreSQL credentials are being sourced directly from Values.env, which are populated by a ConfigMap. Storing secrets like database credentials in ConfigMaps is a critical security risk as they are not encrypted and are easily accessible. You should use Kubernetes Secrets and reference them here using valueFrom.secretKeyRef.

        env:
        - name: POSTGRES_USER
          valueFrom:
            secretKeyRef:
              name: {{ .Values.env.SECRETSTORE_SECRET_NAME }}
              key: PSQL_USER
        - name: POSTGRES_PASSWORD
          valueFrom:
            secretKeyRef:
              name: {{ .Values.env.SECRETSTORE_SECRET_NAME }}
              key: PSQL_PASSWORD
        - name: POSTGRES_DB
          value: {{ .Values.env.PSQL_DB }}

[gemini-code-assist]

The Redis password is being passed as a command-line argument from a value in Values.env, which is sourced from a ConfigMap. This is a critical security vulnerability as it exposes the password in plain text. You should mount the password from a Kubernetes Secret and reference it as an environment variable.

        command: ["redis-server", "--requirepass", "$(REDIS_PASSWORD)"]
        env:
        - name: REDIS_PASSWORD
          valueFrom:
            secretKeyRef:
              name: {{ .Values.env.SECRETSTORE_SECRET_NAME }}
              key: REDIS_PASSWORD

[gemini-code-assist]

The name and version attributes are duplicated in both AppConfig and SecretsConfig. To avoid repetition and improve maintainability, you can extract these common attributes.

Suggested change

	class AppConfig(BaseAppConfig):
	name: str = __version__.split("@")[0]
	version: str = __version__.split("@")[-1]
	description: str = ""
	api_root: str = ""
	# Base Directory
	base_dir: DirectoryPath = Path(__file__).parent.parent.parent.resolve()
	# BudServe Gateway Configuration
	bud_gateway_base_url: str = Field(..., alias="BUD_GATEWAY_BASE_URL")
	bud_default_model_name: str = Field(..., alias="BUD_DEFAULT_MODEL_NAME")
	class SecretsConfig(BaseSecretsConfig):
	name: str = __version__.split("@")[0]
	version: str = __version__.split("@")[-1]
	app_name, app_version = __version__.split("@")
	class AppConfig(BaseAppConfig):
	name: str = app_name
	version: str = app_version
	description: str = ""
	api_root: str = ""
	# Base Directory
	base_dir: DirectoryPath = Path(__file__).parent.parent.parent.resolve()
	# BudServe Gateway Configuration
	bud_gateway_base_url: str = Field(..., alias="BUD_GATEWAY_BASE_URL")
	bud_default_model_name: str = Field(..., alias="BUD_DEFAULT_MODEL_NAME")
	class SecretsConfig(BaseSecretsConfig):
	name: str = app_name
	version: str = app_version

[gemini-code-assist]

Relying on string matching for error messages like "Exceeded maximum retries" is fragile. If the underlying library (pydantic-ai) changes this message in a future version, your error handling will break. It would be more robust to catch a more specific exception type if the library provides one for this scenario.

[gemini-code-assist]

Catching a generic RuntimeError and checking its message string for "asynchronous generator" is not a robust way to handle this specific error. This could unintentionally catch other unrelated RuntimeError exceptions. If possible, find a more specific exception to catch or a safer way to check the state of the generator before calling anext().

[gemini-code-assist]

The error message for a missing APP_NAME suggests using the --app-name flag, but this flag is not parsed by the script. The argument parsing logic should be updated to include this flag, or the error message should be corrected to reflect the available options.

[gemini-code-assist]

The REDIS_PORT environment variable is used here but is not defined in the .env.sample file. While start_dev.sh calculates it, this makes the docker-compose file less portable and harder to understand. It would be clearer to define REDIS_PORT directly in the .env.sample file.