llm-evaluation-framework topic

Test your prompts, agents, and RAGs. Red teaming, pentesting, and vulnerability scanning for LLMs. Compare performance of GPT, Claude, Gemini, Llama, and more. Simple declarative configs with command...

promptfoo

The LLM Evaluation Framework

confident-ai

Python SDK for experimenting, testing, evaluating & monitoring LLM-powered applications - Parea AI (YC S23)

parea-ai

Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

msoedov

The official evaluation suite and dynamic data release for MixEval.

JinjieNi

[ACL'24] A Knowledge-grounded Interactive Evaluation Framework for Large Language Models

zhuohaoyu

FM-Leaderboard-er allows you to create leaderboard to find the best LLM/prompt for your own business use case based on your data, task, prompts

aws-samples

Realign is a testing and simulation framework for AI applications.

honeyhiveai

An easy python package to run quick basic QA evaluations. This package includes standardized QA evaluation metrics and semantic evaluation metrics: Black-box and Open-Source large language model promp...

zli12321

MIT-licensed Framework for LLMs, RAGs, Chatbots testing. Configurable via YAML and integrable into CI pipelines for automated testing.

Addepto