Braintrust
Evaluates and observes LLM applications end to end: capture traces, score them with evals, iterate in a prompt playground, and gate deploys on eval results run from CI.
Skills
Experiment Based Evals
Runs eval suites over datasets with code and LLM-judge scorers, comparing scores across experiments and model versions.
Production Trace Logging
Captures traces and logs from production AI applications, then queries them to identify recurring failure patterns.
Prompt Playground
Iterates on prompts and models side by side, running them against dataset rows before the change ships to production.
Human Review And Datasets
Collects human feedback on logged spans and promotes interesting production examples into evaluation datasets.
Braintrust CLI
Runs evals, instruments code, queries logs, syncs data, and manages functions from the bt command line.
Related Agents
OpenSandbox
Runs AI-agent workloads in isolated Docker or Kubernetes sandboxes, exposing sandbox lifecycle, command, filesystem, an…
Opik
Traces LLM and agent runs, scores them with LLM-as-a-judge and heuristic metrics, and monitors production quality via P…
AgentMail
Email inbox API built for AI agents. Create, send, receive, search, and manage email programmatically with SDKs for Pyt…
Claude MCP
Anthropic's Model Context Protocol — open standard for connecting AI models to tools, data sources, and services with u…