Browse Agents
Task
I need to test and evaluate agents
Trace, test and score LLM apps and agents before and after they ship.
Picked from listings that mention evals, evaluation, llm observability, tracing or prompt testing in their name, description, tags or skills; those in Infrastructure & Ops, Code & DevTools or Orchestration, or with API access, rank higher. About this task →
29 agents fit this task
13 Official · 28 Verified · 5 No auth needed · 27 API · 9 MCP · 20 CLI
Langfuse
API · CLIOpen-source LLM observability and analytics. Trace, evaluate, and debug LLM apps with prompt versioning, score…
Future AGI
API · CLIEvaluate, observe, and improve LLM and agent applications from one self-hostable platform — tracing, evals…
Microsoft PromptFlow
API · CLIBuild, evaluate, and deploy LLM applications as visual DAG flows — chain prompts, Python, and tools, then run batch…
OpenLLMetry
APIInstruments LLM and agent applications with OpenTelemetry-native tracing, exporting spans for prompts, tool calls, and…
Latitude
API · CLI · MCPTraces, scores, and evaluates production AI agents via OpenTelemetry and SDKs, groups failures into signals, and…
MCPJam
API · CLI · MCPTests and evaluates MCP servers and MCP apps for developers, with a playground, OAuth debugger, cross-client evals, and…
LLM Space
—Prototypes, traces, replays, and evaluates LLM agents in a local macOS desktop app, and exports threads as runnable…
Opik
API · MCP · CLITraces LLM and agent runs, scores them with LLM-as-a-judge and heuristic metrics, and monitors production quality via…
Datadog LLM Observability
API · MCP · CLIEnd-to-end monitoring for LLM applications — trace prompts, evaluate responses, track costs, and debug AI pipelines in…
Helicone
APIMonitors LLM and agent traffic through a one-line proxy integration, logging requests, costs and latency with caching…
Arize Phoenix
API · CLITraces, evaluates and debugs LLM and agent applications from an open-source observability platform built on…
Google Agents CLI
CLITurns any coding assistant into an expert at creating, evaluating, and deploying AI agents on Google Cloud. An open CLI…
Laminar
API · CLI · MCPTrace, evaluate, and debug AI agents with open-source observability — captures LLM calls, tool calls, and sub-agent…
Braintrust
API · CLI · MCPEvaluates and observes LLM applications end to end: capture traces, score them with evals, iterate in a prompt…
LangChain / LangSmith
API · MCP · CLIFramework for building LLM applications with chains, agents, and tool use — plus LangSmith for tracing, evaluation, and…
AgentScope
CLI · APIBuilds multi-agent applications with a Python framework focused on transparency, observability, and tracing. Includes a…
Armature
API · MCPProduct analytics and evaluation for agent sessions running against MCP servers, tracking tool calls, failures, and…
Genkit
API · CLIBuilds AI features and agent flows in JavaScript, Go or Python with typed flows, tool calling and a local dev UI for…
Cursor Agent SDK
API · CLILets developers define, test, evaluate, and deploy agents as Markdown and TypeScript code, on Cursor-managed hosting or…
DeepEval
CLI · APIUnit-tests LLM and agent outputs in a pytest-style workflow, scoring answer relevancy, faithfulness, hallucination and…
AgentOps
APIObservability platform purpose-built for AI agents. Track agent sessions, LLM calls, tool use, and costs. Integrates…
promptfoo
CLI · APITests and red-teams LLM apps, agents and RAG pipelines from declarative config, scanning for prompt injection…
NVIDIA NeMo Agent Toolkit
API · CLI · MCPConnects, profiles, evaluates, and optimizes agent workflows across frameworks like LangChain, LlamaIndex, and CrewAI…
VoltAgent
API · CLIBuilds TypeScript AI agents with tools, memory, and sub-agent supervision, paired with a local observability console…
Hatchet
API · CLIDurable task orchestration engine for AI agents — parallel workloads, automatic retries, concurrency controls, and…
HyperProbe
APIAgents that perform read-only production debugging, tracing failures across services without mutating state. Y…
Cloudflare AI Gateway
APIProxies requests to OpenAI, Anthropic, Google, Workers AI, and other model providers, adding analytics, logging…
Portkey
APIAI gateway with unified API for 200+ LLMs, plus observability, caching, fallbacks, and cost management. Drop-in…
Vercel AI Gateway
APIRoutes model calls from any app or coding agent across providers with one API key, adding fallbacks, request logs…