Whistleblower is a offensive security tool for testing against system prompt leakage and capability discovery of an AI application exposed through API. Built for AI engineers, security researchers and folks who want to know what's going on inside the LLM-based app they use daily

PromptMe is an educational project that showcases security vulnerabilities in large language models (LLMs) and their web integrations. It includes 10 hands-on challenges inspired by the OWASP LLM Top 10, demonstrating how these vulnerabilities can be discovered and exploited in real-world scenarios.

Resk is a robust Python library designed to enhance security and manage context when interacting with LLMs. It provides a protective layer for API calls, safeguarding against common vulnerabilities and ensuring optimal performance. And safe layer again Prompt Injection.

Utterly unelegant prompts for local LLMs, with scary results.

Lakera Gandalf AI challenge's step by step walkthrough, showcasing real-world prompt injection techniques and LLM security insights.

Data Analysis of the results of llmail-inject challenge

A comprehensive reference for securing Large Language Models (LLMs). Covers OWASP GenAI Top-10 risks, prompt injection, adversarial attacks, real-world incidents, and practical defenses. Includes catalogs of red-teaming tools, guardrails, and mitigation strategies to help developers, researchers, and security teams deploy AI responsibly.

Proof of Concept (PoC) demonstrating prompt injection vulnerability in AI code assistants (like Copilot) using hidden Unicode characters within instruction files (copilot-instructions.md). Highlights risks of using untrusted instruction templates. For educational/research purposes only.

FRACTURED-SORRY-Bench: This repository contains the code and data for the creating an Automated Multi-shot Jailbreak framework, as described in our paper.

A secure database server for storing LLM memories with comprehensive content validation. This server validates content for malicious patterns including hate speech, prompt injection, and illegal content before allowing storage.

Bidirectional Security Framework for Human/LLM Interfaces - RC9-FPR4 baseline frozen. ASR: 2.76% (Wilson Upper 3.59%, GATE PASS). FPR stratified: doc_with_codefence 0.79% Upper (GATE PASS), pure_doc 4.69% Upper (N=303). Tests: 833/853 (97.7%), MyPy clean, CI GREEN. Shadow deployment ready. ktzzzn

A prototype defense against prompt-based attacks with real-time threat assessment.

이모지 스머글링, 이모지 이베이젼 겉 핥기

🛠️ Explore large language models through hands-on projects and tutorials to enhance your understanding and practical skills in natural language processing.

🔍 Analyze system prompts in large language models to understand design principles and enhance AI application effectiveness.

🛡️ Explore tools for securing Large Language Models, uncovering their strengths and weaknesses in the realm of offensive and defensive security.