Skip to content

Conversation

@gehad-alaa-abaas
Copy link

DLP Benchmark Module for AI Security & Privacy Guide

Overview

This pull request introduces a new module for benchmarking Data Loss Prevention (DLP) in AI systems.
The module provides tools to evaluate how effectively AI guardrails prevent sensitive data leakage.

What’s Included

  • Benchmark Script
    A flexible Python tool for evaluating DLP guardrails, supporting both batch and single-input validation.

  • Synthetic Dataset
    Over 20 realistic prompts and contexts covering personal information, secrets, and confidential data, designed to simulate real-world scenarios.

  • Configurable Guardrails
    Easy integration with different AI guardrail modules via a YAML config file.

  • Documentation
    Updated README.md with usage instructions, advanced options, and guidance for extending the benchmark.

Why This Matters

AI systems are increasingly handling sensitive data, and robust DLP is essential for privacy and compliance.

This module helps teams:

  • Assess and improve their AI guardrails.
  • Run repeatable and transparent benchmarks.
  • Build confidence in handling sensitive information safely.

@robvanderveer
Copy link
Collaborator

Hi Gehad. This is really nice work. Would you agree that the best approach to manage this work is let it have its own repo and then work with the Exchange Testing team to get it discussed and referred to in our testing guide? If so, I welcome you to reach out to Behnaz Karimi on OWASP Slack to make this happen. Thanks!

@gehad-alaa-abaas
Copy link
Author

Hi @robvanderveer
Thank you! I agree that creating a dedicated repository for this work is the best approach. I’ll reach out to @behyka Behnaz Karimi on OWASP Slack to coordinate with the Exchange Testing team and ensure it’s discussed and referenced in the testing guide.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants