Merge branch 'main' of https://github.com/stanfordnlp/dspy into stanfordnlp-main

Author: GFarnon

.gitignore

@@ -47,3 +47,5 @@ assertion.log
 *.log
 *.db
 /.devcontainer/.personalization.sh
+
+.mypy_cache

README.md

@@ -58,18 +58,25 @@ Ditto! **DSPy** gives you the right general-purpose modules (e.g., `ChainOfThoug
 
 All you need is:
 
-```
+```bash
 pip install dspy-ai
 ```
 
+To install the very latest from `main`:
+
+```bash
+pip install git+https://github.com/stanfordnlp/dspy.git
+````
+
 Or open our intro notebook in Google Colab: [<img align="center" src="https://colab.research.google.com/assets/colab-badge.svg" />](https://colab.research.google.com/github/stanfordnlp/dspy/blob/main/intro.ipynb)
 
 By default, DSPy installs the latest `openai` from pip. However, if you install old version before OpenAI changed their API `openai~=0.28.1`, the library will use that just fine. Both are supported.
 
-For the optional (alphabetically sorted) [Chromadb](https://github.com/chroma-core/chroma), [Qdrant](https://github.com/qdrant/qdrant), [Marqo](https://github.com/marqo-ai/marqo), Pinecone, or [Weaviate](https://github.com/weaviate/weaviate) retrieval integration(s), include the extra(s) below:
+For the optional (alphabetically sorted) [Chromadb](https://github.com/chroma-core/chroma), [Qdrant](https://github.com/qdrant/qdrant), [Marqo](https://github.com/marqo-ai/marqo), Pinecone, [Weaviate](https://github.com/weaviate/weaviate),
+or [Milvus](https://github.com/milvus-io/milvus) retrieval integration(s), include the extra(s) below:
 
 ```
-pip install dspy-ai[chromadb]  # or [qdrant] or [marqo] or [mongodb] or [pinecone] or [weaviate]
+pip install dspy-ai[chromadb]  # or [qdrant] or [marqo] or [mongodb] or [pinecone] or [weaviate] or [milvus]
 ```
 
 ## 2) Documentation
@@ -94,10 +101,11 @@ The DSPy documentation is divided into **tutorials** (step-by-step illustration
 
 - [DSPy talk at ScaleByTheBay Nov 2023](https://www.youtube.com/watch?v=Dt3H2ninoeY).
 - [DSPy webinar with MLOps Learners](https://www.youtube.com/watch?v=im7bCLW2aM4), a bit longer with Q&A.
-- Hands-on Overviews of DSPy by the community: [DSPy Explained! by Connor Shorten](https://www.youtube.com/watch?v=41EfOY0Ldkc), [DSPy explained by code_your_own_ai](https://www.youtube.com/watch?v=ycfnKPxBMck)
+- Hands-on Overviews of DSPy by the community: [DSPy Explained! by Connor Shorten](https://www.youtube.com/watch?v=41EfOY0Ldkc), [DSPy explained by code_your_own_ai](https://www.youtube.com/watch?v=ycfnKPxBMck), [DSPy Crash Course by AI Bites](https://youtu.be/5-zgASQKkKQ?si=3gnmVouT5_rpk_nu)
 - Interviews: [Weaviate Podcast in-person](https://www.youtube.com/watch?v=CDung1LnLbY), and you can find 6-7 other remote podcasts on YouTube from a few different perspectives/audiences.
 - **Tracing in DSPy** with Arize Phoenix: [Tutorial for tracing your prompts and the steps of your DSPy programs](https://colab.research.google.com/github/Arize-ai/phoenix/blob/main/tutorials/tracing/dspy_tracing_tutorial.ipynb)
 - [DSPy: Not Your Average Prompt Engineering](https://jina.ai/news/dspy-not-your-average-prompt-engineering), why it's crucial for future prompt engineering, and yet why it is challenging for prompt engineers to learn.
+- **Tracing & Optimization Tracking in DSPy** with Parea AI: [Tutorial on tracing & evaluating a DSPy RAG program](https://docs.parea.ai/tutorials/dspy-rag-trace-evaluate/tutorial)
 
 ### B) Guides
 
@@ -129,22 +137,29 @@ You can find other examples tweeted by [@lateinteraction](https://twitter.com/la
 
 **Some other examples (not exhaustive, feel free to add more via PR):**
 
+
+- [DSPy Optimizers Benchmark on a bunch of different tasks, by Michael Ryan](https://github.com/stanfordnlp/dspy/tree/main/testing/tasks)
+- [Sophisticated Extreme Multi-Class Classification, IReRa, by Karel D’Oosterlinck](https://github.com/KarelDO/xmc.dspy)
+- [Haize Lab's Red Teaming with DSPy](https://blog.haizelabs.com/posts/dspy/) and see [their DSPy code](https://github.com/haizelabs/dspy-redteam)
 - Applying DSPy Assertions
   - [Long-form Answer Generation with Citations, by Arnav Singhvi](https://colab.research.google.com/github/stanfordnlp/dspy/blob/main/examples/longformqa/longformqa_assertions.ipynb)
   - [Generating Answer Choices for Quiz Questions, by Arnav Singhvi](https://colab.research.google.com/github/stanfordnlp/dspy/blob/main/examples/quiz/quiz_assertions.ipynb)
   - [Generating Tweets for QA, by Arnav Singhvi](https://colab.research.google.com/github/stanfordnlp/dspy/blob/main/examples/tweets/tweets_assertions.ipynb)
 - [Compiling LCEL runnables from LangChain in DSPy](https://github.com/stanfordnlp/dspy/blob/main/examples/tweets/compiling_langchain.ipynb)
 - [AI feedback, or writing LM-based metrics in DSPy](https://github.com/stanfordnlp/dspy/blob/main/examples/tweets/tweet_metric.py)
-- [DSPy Optimizers Benchmark on a bunch of different tasks, by Michael Ryan](https://github.com/stanfordnlp/dspy/tree/main/testing/tasks)
+- [DSPy Optimizers Benchmark on a bunch of different tasks, by Michael Ryan](https://github.com/stanfordnlp/dspy/tree/main/testing/README.md)
 - [Indian Languages NLI with gains due to compiling by Saiful Haq](https://github.com/saifulhaq95/DSPy-Indic/blob/main/indicxlni.ipynb)
-- [Sophisticated Extreme Multi-Class Classification, IReRa, by Karel D’Oosterlinck](https://github.com/KarelDO/xmc.dspy)
 - [DSPy on BIG-Bench Hard Example, by Chris Levy](https://drchrislevy.github.io/posts/dspy/dspy.html)
 - [Using Ollama with DSPy for Mistral (quantized) by @jrknox1977](https://gist.github.com/jrknox1977/78c17e492b5a75ee5bbaf9673aee4641)
-- [Using DSPy, "The Unreasonable Effectiveness of Eccentric Automatic Prompts" (paper) by VMware's Rick Battle & Teja Gollapudi, and interview at TheRegister](https://www.theregister.com/2024/02/22/prompt_engineering_ai_models/)
+- [Using DSPy, "The Unreasonable Effectiveness of Eccentric Automatic Prompts" (paper) by VMware's Rick Battle & Teja Gollapudi](https://arxiv.org/abs/2402.10949), and [interview at TheRegister](https://www.theregister.com/2024/02/22/prompt_engineering_ai_models/)
+- [Optimizing Performance of Open Source LM for Text-to-SQL using DSPy and vLLM, by Juan Ovalle](https://github.com/jjovalle99/DSPy-Text2SQL)
 - Typed DSPy (contributed by [@normal-computing](https://github.com/normal-computing))
   - [Using DSPy to train Gpt 3.5 on HumanEval by Thomas Ahle](https://github.com/stanfordnlp/dspy/blob/main/examples/functional/functional.ipynb)
   - [Building a chess playing agent using DSPy by Franck SN](https://medium.com/thoughts-on-machine-learning/building-a-chess-playing-agent-using-dspy-9b87c868f71e)
 
+
+TODO: Add links to the state-of-the-art results by the University of Toronto on Clinical NLP, on Theory of Mind (ToM) by Plastic Labs, and the DSPy pipeline from Replit.
+
 There are also recent cool examples at [Weaviate's DSPy cookbook](https://github.com/weaviate/recipes/tree/main/integrations/dspy) by Connor Shorten. [See tutorial on YouTube](https://www.youtube.com/watch?v=CEuUG4Umfxs).
 
 ## 3) Syntax: You're in charge of the workflow—it's free-form Python code!
@@ -411,7 +426,7 @@ See [CONTRIBUTING.md](CONTRIBUTING.md) for a quickstart guide to contributing to
 
 **DSPy** is led by **Omar Khattab** at Stanford NLP with **Chris Potts** and **Matei Zaharia**.
 
-Key contributors and team members include **Arnav Singhvi**, **Krista Opsahl-Ong**, **Michael Ryan**, **Karel D'Oosterlinck**, **Shangyin Tan**, **Manish Shetty**, **Paridhi Maheshwari**, **Keshav Santhanam**, **Sri Vardhamanan**, **Eric Zhang**, **Hanna Moazam**, **Thomas Joshi**, **Saiful Haq**, **Ashutosh Sharma**, and **Herumb Shandilya**.
+Key contributors and team members include **Arnav Singhvi**, **Krista Opsahl-Ong**, **Michael Ryan**, **Cyrus Nouroozi**, **Kyle Caverly**, **Amir Mehr**, **Karel D'Oosterlinck**, **Shangyin Tan**, **Manish Shetty**, **Herumb Shandilya**, **Paridhi Maheshwari**, **Keshav Santhanam**, **Sri Vardhamanan**, **Eric Zhang**, **Hanna Moazam**, **Thomas Joshi**, **Saiful Haq**, and **Ashutosh Sharma**.
 
 **DSPy** includes important contributions from **Rick Battle** and **Igor Kotenkov**. It reflects discussions with **Peter Zhong**, **Haoze He**, **Lisa Li**, **David Hall**, **Ashwin Paranjape**, **Heather Miller**, **Chris Manning**, **Percy Liang**, and many others.
 

docs/api/language_model_clients/AzureOpenAI.md

@@ -14,6 +14,8 @@ lm = dspy.AzureOpenAI(api_base='...', api_version='2023-12-01-preview', model='g
 
 The constructor initializes the base class `LM` and verifies the provided arguments like the `api_provider`, `api_key`, and `api_base` to set up OpenAI request retrieval through Azure. The `kwargs` attribute is initialized with default values for relevant text generation parameters needed for communicating with the GPT API, such as `temperature`, `max_tokens`, `top_p`, `frequency_penalty`, `presence_penalty`, and `n`.
 
+Azure requires that the deployment id of the Azure deployment to be also provided using the argument `deployment_id`.
+
 ```python
 class AzureOpenAI(LM):
     def __init__(
@@ -53,4 +55,4 @@ After generation, the completions are post-processed based on the `model_type` p
 - `**kwargs`: Additional keyword arguments for completion request.
 
 **Returns:**
-- `List[Dict[str, Any]]`: List of completion choices.
+- `List[Dict[str, Any]]`: List of completion choices.

docs/api/language_model_clients/Google_VertexAI.md

@@ -0,0 +1,52 @@
+# GoogleVertexAI Usage Guide
+
+This guide provides instructions on how to use the `GoogleVertexAI` class to interact with Google Vertex AI's API for text and code generation.
+
+## Requirements
+
+- Python 3.10 or higher.
+- The `vertexai` package installed, which can be installed via pip.
+- A Google Cloud account and a configured project with access to Vertex AI.
+
+## Installation
+
+Ensure you have installed the `vertexai` package along with other necessary dependencies:
+
+```bash
+pip install dspy-ai[google-vertex-ai]
+```
+
+## Configuration
+
+Before using the `GoogleVertexAI` class, you need to set up access to Google Cloud:
+
+1. Create a project in Google Cloud Platform (GCP).
+2. Enable the Vertex AI API for your project.
+3. Create authentication credentials and save them in a JSON file.
+
+## Usage
+
+Here's an example of how to instantiate the `GoogleVertexAI` class and send a text generation request:
+
+```python
+from dsp.modules import GoogleVertexAI  # Import the GoogleVertexAI class
+
+# Initialize the class with the model name and parameters for Vertex AI
+vertex_ai = GoogleVertexAI(
+    model_name="text-bison@002",
+    project="your-google-cloud-project-id",
+    location="us-central1",
+    credentials="path-to-your-service-account-file.json"
+)
+```
+
+## Customizing Requests
+
+You can customize requests by passing additional parameters such as `temperature`, `max_output_tokens`, and others supported by the Vertex AI API. This allows you to control the behavior of the text generation.
+
+## Important Notes
+
+- Make sure you have correctly set up access to Google Cloud to avoid authentication issues.
+- Be aware of the quotas and limits of the Vertex AI API to prevent unexpected interruptions in service.
+
+With this guide, you're ready to use `GoogleVertexAI` for interacting with Google Vertex AI's text and code generation services.

docs/api/language_model_clients/Watsonx.md

@@ -0,0 +1,55 @@
+# Watsonx Usage Guide
+
+This guide provides instructions on how to use the `Watsonx` class to interact with  IBM Watsonx.ai API for text and code generation.
+
+## Requirements
+
+- Python 3.10 or higher.
+- The `ibm-watsonx-ai` package installed, which can be installed via pip.
+- An IBM Cloud account and a Watsonx configured project.
+
+## Installation
+
+Ensure you have installed the `ibm-watsonx-ai` package along with other necessary dependencies:
+
+## Configuration
+
+Before using the `Watsonx` class, you need to set up access to IBM Cloud:
+
+1. Create an IBM Cloud account
+2. Enable a Watsonx service from the catalog
+3. Create a new project and associate a Watson Machine Learning service instance.
+4. Create an IAM authentication credentials and save them in a JSON file.
+
+## Usage
+
+Here's an example of how to instantiate the `Watsonx` class and send a generation request:
+
+```python
+import dspy
+
+''' Initialize the class with the model name and parameters for Watsonx.ai
+    You can choose between many different models:
+    * (Mistral) mistralai/mixtral-8x7b-instruct-v01
+    * (Meta) meta-llama/llama-3-70b-instruct
+    * (IBM) ibm/granite-13b-instruct-v2
+    * and many others.
+'''
+watsonx=dspy.Watsonx(
+    model='mistralai/mixtral-8x7b-instruct-v01',
+    credentials={
+        "apikey": "your-api-key",
+        "url": "https://us-south.ml.cloud.ibm.com"
+    },
+    project_id="your-watsonx-project-id",
+    max_new_tokens=500,
+    max_tokens=1000
+    )
+
+dspy.settings.configure(lm=watsonx)
+```
+
+## Customizing Requests
+
+You can customize requests by passing additional parameters such as `decoding_method`,`max_new_tokens`, `stop_sequences`, `repetition_penalty`, and others supported by the Watsonx.ai API. This allows you to control the behavior of the generation.
+Refer to [`ibm-watsonx-ai library`](https://ibm.github.io/watsonx-ai-python-sdk/index.html) documentation.

docs/api/language_model_clients/aws_models.md

@@ -0,0 +1,86 @@
+---
+sidebar_position: 9
+---
+
+# dspy.AWSMistral, dspy.AWSAnthropic, dspy.AWSMeta
+
+### Usage
+
+```python
+# Notes:
+# 1. Install boto3 to use AWS models.
+# 2. Configure your AWS credentials with the AWS CLI before using these models
+
+# initialize the bedrock aws provider
+bedrock = dspy.Bedrock(region_name="us-west-2")
+# For mixtral on Bedrock
+lm = dspy.AWSMistral(bedrock, "mistral.mixtral-8x7b-instruct-v0:1", **kwargs)
+# For haiku on Bedrock
+lm = dspy.AWSAnthropic(bedrock, "anthropic.claude-3-haiku-20240307-v1:0", **kwargs)
+# For llama2 on Bedrock
+lm = dspy.AWSMeta(bedrock, "meta.llama2-13b-chat-v1", **kwargs)
+
+# initialize the sagemaker aws provider
+sagemaker = dspy.Sagemaker(region_name="us-west-2")
+# For mistral on Sagemaker
+# Note: you need to create a Sagemaker endpoint for the mistral model first
+lm = dspy.AWSMistral(sagemaker, "<YOUR_MISTRAL_ENDPOINT_NAME>", **kwargs)
+
+```
+
+### Constructor
+
+The `AWSMistral` constructor initializes the base class `AWSModel` which itself inherits from the `LM` class.
+
+```python
+class AWSMistral(AWSModel):
+    """Mistral family of models."""
+
+    def __init__(
+        self,
+        aws_provider: AWSProvider,
+        model: str,
+        max_context_size: int = 32768,
+        max_new_tokens: int = 1500,
+        **kwargs
+    ) -> None:
+```
+
+**Parameters:**
+- `aws_provider` (AWSProvider): The aws provider to use. One of `dspy.Bedrock` or `dspy.Sagemaker`.
+- `model` (_str_): Mistral AI pretrained models. For Bedrock, this is the Model ID in https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html#model-ids-arns. For Sagemaker, this is the endpoint name.
+- `max_context_size` (_Optional[int]_, _optional_): Max context size for this model. Defaults to 32768.
+- `max_new_tokens` (_Optional[int]_, _optional_): Max new tokens possible for this model. Defaults to 1500.
+- `**kwargs`: Additional language model arguments to pass to the API provider.
+
+### Methods
+
+```python
+def _format_prompt(self, raw_prompt: str) -> str:
+```
+This function formats the prompt for the model. Refer to the model card for the specific formatting required.
+
+<br/>
+
+```python
+def _create_body(self, prompt: str, **kwargs) -> tuple[int, dict[str, str | float]]:
+```
+This function creates the body of the request to the model. It takes the prompt and any additional keyword arguments and returns a tuple of the number of tokens to generate and a dictionary of keys including the prompt used to create the body of the request.
+
+<br/>
+
+```python
+def _call_model(self, body: str) -> str:
+```
+This function calls the model using the provider `call_model()` function and extracts the generated text (completion) from the provider-specific response.
+
+<br/>
+
+The above model-specific methods are called by the `AWSModel::basic_request()` method, which is the main method for querying the model. This method takes the prompt and any additional keyword arguments and calls the `AWSModel::_simple_api_call()` which then delegates to the model-specific `_create_body()` and `_call_model()` methods to create the body of the request, call the model and extract the generated text.
+
+
+Refer to [`dspy.OpenAI`](https://dspy-docs.vercel.app/api/language_model_clients/OpenAI) documentation for information on the `LM` base class functionality.
+
+<br/>
+
+`AWSAnthropic` and `AWSMeta` work exactly the same as `AWSMistral`.

docs/api/language_model_clients/aws_providers.md

@@ -0,0 +1,53 @@
+---
+sidebar_position: 9
+---
+
+# dspy.Bedrock, dspy.Sagemaker
+
+### Usage
+
+The `AWSProvider` class is the base class for the AWS providers - `dspy.Bedrock` and `dspy.Sagemaker`. An instance of one of these providers is passed to the constructor when creating an instance of an AWS model class (e.g., `dspy.AWSMistral`) that is ultimately used to query the model.
+
+```python
+# Notes:
+# 1. Install boto3 to use AWS models.
+# 2. Configure your AWS credentials with the AWS CLI before using these models
+
+# initialize the bedrock aws provider
+bedrock = dspy.Bedrock(region_name="us-west-2")
+
+# initialize the sagemaker aws provider
+sagemaker = dspy.Sagemaker(region_name="us-west-2")
+```
+
+### Constructor
+
+The `Bedrock` constructor initializes the base class `AWSProvider`.
+
+```python
+class Bedrock(AWSProvider):
+    """This class adds support for Bedrock models."""
+
+    def __init__(
+        self,
+        region_name: str,
+        profile_name: Optional[str] = None,
+        batch_n_enabled: bool = False,   # This has to be setup manually on Bedrock.
+    ) -> None:
+```
+
+**Parameters:**
+- `region_name` (str): The AWS region where this LM is hosted.
+- `profile_name` (str, optional): boto3 credentials profile.
+- `batch_n_enabled` (bool): If False, call the LM N times rather than batching.
+
+### Methods
+
+```python
+def call_model(self, model_id: str, body: str) -> str:
+```
+This function implements the actual invocation of the model on AWS using the boto3 provider.
+
+<br/>
+
+`Sagemaker` works exactly the same as `Bedrock`.