FAQ Chatbot CLI

A command-line interface (CLI) chatbot for FAQs, leveraging Azure OpenAI's embedding and chat models. This project is designed to provide a simple and effective way to interact with FAQs using natural language processing.

This project is built with Python and can be run in a Docker container or as a standalone Python application. It supports Langfuse for observability and tracing.

Why Azure OpenAI?

I chose to use Azure OpenAI because it provides a powerful and scalable solution for natural language processing tasks, including embeddings and chat models, while ensuring compliance with data residency requirements.

Selection to ensure compliance with data residency requirements and gdpr in the european union:

• Resource Region: swedish central (westeurope is also possible)
• Deployment Type: Standard deployment type

With deploying the resources in the swedish central region together with selecting the Standard deployment type, I can ensure compliance with data residency requirements for sensitive information and gdpr. Data stored at rest and also data processed for inference will not leave the selected region.

Which models I used?

• Embedding Model: text-embedding-ada-002
• Chat Model: gpt-35-turbo

Why these models?

• The text-embedding-ada-002 model is efficient for generating embeddings, cost-effective, and provides good performance for similarity search tasks. Also text-embedding-3 models seems to have issues with FAISS similarity search with Langchain.
• The gpt-35-turbo model is a powerful chat model that provides high-quality responses for simple FAQ tasks. It is cost-effective and widely used in production applications.

Installation and Environment Setup

1. Prerequisites

• Python 3.10 or higher
• Docker and Docker Compose
• Azure Embedding Model deployed
• Azure Chat Model deployed
• Azure OpenAI Endpoint and Key

2. Clone the Repository

git clone git@github.com:MSWagner/faq-cli-chatbot.git
cd faq-cli-chatbot

3. Create a .env file

Copy the .env.example to .env and fill in the required environment variables:

cp .env.example .env

• Ensure you have the following variables set in your .env file:

  • AZURE_OPENAI_ENDPOINT
  • AZURE_OPENAI_KEY
  • EMBEDDING_MODEL
  • EMBEDDING_API_VERSION
  • CHAT_MODEL
  • CHAT_API_VERSION

• (Optional) variables for Langfuse integration (local setup described in the next sections):

  • LANGFUSE_SECRET_KEY
  • LANGFUSE_PUBLIC_KEY
  • LANGFUSE_HOST (if you want to use a remote Langfuse instance or start the chatbot outside of a docker container)
  • LANGFUSE_SYSTEM_PROMPT (if you want to use a system prompt from Langfuse)

Hints:

• use AZURE_OPENAI_ENDPOINT in the format https://<your-resource-name>.openai.azure.com/
• you can find the model names and versions in the Azure AI Foundary Studio under "Deployments".

Chatbot with Docker Compose

Start the chatbot

docker compose run --rm faq-chatbot

Hint:

• If you want to run the chatbot with remote Langfuse, ensure you have the Langfuse environment variables set in your .env file.

Chatbot with local Langfuse

1. First Time Setup

Build the Docker container with Langfuse:

docker compose -f 'docker-compose.langfuse.yml' up -d --build

Do the following steps in the Langfuse WebUI (http://localhost:3000):

• Create a new User (Signup) and create a new Organisation and Project.
• Create an API Key in the Langfuse WebUI and copy the LANGFUSE_PUBLIC_KEY and LANGFUSE_SECRET_KEY to your .env file.
• (Optional) Create a System Prompt in Langfuse and copy the LANGFUSE_SYSTEM_PROMPT to your .env file.

Rebuild the Docker container with the new environment variables:

source .env
docker compose -f 'docker-compose.langfuse.yml' up -d --build

2. Start the chatbot with Langfuse

docker compose -f 'docker-compose.langfuse.yml' run --rm faq-chatbot

Chatbot Python CLI

1. Create a Virtual Environment (recommended)

Create a virtual environment to isolate dependencies:

python -m venv .venv
source .venv/bin/activate  # For Windows: .venv\Scripts\activate

2. Install Dependencies

pip install -e .  # Installs the project in editable mode, reading from pyproject.toml

3. Start the Chatbot

python -m faqbot.main

# Force rebuild embeddings
python -m faqbot.main -f

# Enable debug mode
python -m faqbot.main -d

# Reload .env variables
python -m faqbot.main -r

4. Run Tests

python -m pytest --rich

License

This project is licensed under the MIT License. See the LICENSE file for details.