Traces LLM and agent runs, scores them with LLM-as-a-judge and heuristic metrics, and monitors production quality via Python and TypeScript SDKs and 60+ integrations.
- Access
- APIMCPCLI
- Auth
- API Key
- Skills
- 5
Skills
Agent Trace Logging
Captures trace trees of LLM calls, tool executions, and retrieval steps via a track decorator or framework integrations.
LLM As Judge Metrics
Scores outputs for hallucination, moderation, answer relevance, and context precision with built-in or custom judge metrics.
Datasets And Experiments
Evaluates applications against versioned datasets as experiments and compares the results in the Opik dashboard.
Production Monitoring Rules
Applies online evaluation rules to live traces and charts feedback scores, trace counts, and token usage over time.
PyTest CI Evaluation
Tests LLM pipelines on every commit through the PyTest integration so evaluation results are recorded per CI run.
Related Agents
Braintrust
API · CLI · MCPEvaluates and observes LLM applications end to end: capture traces, score them with evals, iterate in a prompt…
Arize Phoenix
API · CLITraces, evaluates and debugs LLM and agent applications from an open-source observability platform built on…
Langfuse
API · CLIOpen-source LLM observability and analytics. Trace, evaluate, and debug LLM apps with prompt versioning, score…
OpenLLMetry
APIInstruments LLM and agent applications with OpenTelemetry-native tracing, exporting spans for prompts, tool calls, and…