Red Teaming python-framework for testing chatbots and GenAI systems

Install 🚀

pip install llamator==3.4.0

Documentation 📚

Documentation Link: https://LLAMATOR-Core.github.io/llamator

Guides ▶️

• LLM Red Teaming with LLAMATOR
• LLAMATOR: Quick Start Guide

Examples 💡

• 📄 RAG bot testing via REST API
• 🧙‍♂️ Gandalf web bot testing via Selenium
• 💬 Telegram bot testing via Telethon
• 📱 WhatsApp bot testing via Selenium
• 🔗 LangChain client testing with custom attack
• 📷 Vision Model testing via LangChain
• 🐋 Running LLAMATOR in Docker

Supported Clients 🛠️

• 🌐 All LangChain clients
• 🧠 OpenAI-like API
• ⚙️ Custom Class (Telegram, WhatsApp, Selenium, etc.)

Unique Features 🌟

• ️🗡 Support for custom attacks and datasets from the user
• 👜 Large selection of attacks on LLMs, RAGs, Agents, VLMs in English and Russian
• 🛡 Custom configuration of chat clients
• 📊 History of attack requests and responses in Excel and CSV format
• 📄 Test report document in DOCX format

OWASP Classification 🔒

• 💉 LLM01: Prompt Injection and Jailbreaks
• 🕵️‍♀️ LLM07: System Prompt Leakage
• 🤥 LLM09: Misinformation
• 💸 LLM10: Unbounded Consumption

Community 🌍

• 💬 Telegram Chat — LLAMATOR | AI Red Team Community
• 🖤 Supported by AI Talent Hub and ITMO University

Contacts 📬

✉️ Email: [email protected]

Enterprise Version 🏢

⚡️Coming Soon⚡️

License 📜

© Roman Neronov, Timur Nizamov, Nikita Ivanov

This project is licensed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International license. See the LICENSE file for details.