switchboard
p

promptfoo

by promptfoo

Tests and red-teams LLM apps, agents and RAG pipelines from declarative config, scanning for prompt injection, jailbreaks and other vulnerabilities in CI.

3
Skills
None
Auth
No
Streaming
No
Push

Skills

Declarative Eval Suites

Runs prompt and agent test cases from a config file, scoring outputs with assertions and model graders.

Automated Red Teaming

Generates adversarial attacks against an application to find prompt injection, jailbreaks and data leakage.

Model Comparison

Compares providers and prompt variants side by side on the same dataset to justify a model choice with data.

SecurityCode & DevToolsllm-evaluationred-teamingprompt-injectionregression-testingci-integrationjailbreak-scanning
Visit Agent

Related Agents

O

OpenSandbox

Runs AI-agent workloads in isolated Docker or Kubernetes sandboxes, exposing sandbox lifecycle, command, filesystem, an…

Infrastructure & OpsCode & DevTools
APIMCPCLI
5 skillsAPI Key
I

IDA Pro MCP

Bridges IDA Pro to LLM clients over MCP, exposing decompilation, disassembly, cross-references, type and stack edits, m…

SecurityCode & DevTools
MCPCLI
5 skillsNone
S

Snyk

Developer security platform with AI-powered vulnerability detection, fix suggestions, and automated security testing ac…

SecurityCode & DevTools
APIMCPCLI
0 skillsAPI Key
S

SWE-agent

Takes a GitHub issue and autonomously fixes it with the language model of your choice. Also runs cybersecurity/CTF task…

Code & DevToolsSecurity
CLIAPI
3 skillsAPI Key
promptfoo
Tests and red-teams LLM apps, agents and RAG pipelines from declarative config, scanning for prompt injection, jailbreaks and other vulnerabilities in CI.
fields
namepromptfoo
providerpromptfoo
urlhttps://github.com/promptfoo/promptfoo
categoriessecurity · code-devtools
accesscli · api
authnone
streamingfalse
pushfalse
verifiedtrue
tagsllm-evaluation, red-teaming, prompt-injection, regression-testing, ci-integration, jailbreak-scanning
skills
eval-suitesDeclarative Eval SuitesRuns prompt and agent test cases from a config file, sc…
red-teamingAutomated Red TeamingGenerates adversarial attacks against an application to…
model-comparisonModel ComparisonCompares providers and prompt variants side by side on …
→ https://github.com/promptfoo/promptfoo