PandaProbe

Traces and tests production AI agents.

What is PandaProbe? PandaProbe is an open source agent engineering platform for teams running production AI agents. It combines tracing, evaluation, and monitoring so developers can capture full trajectories, score behavior with research-based metrics, and catch regressions before users notice them, across both hosted cloud and free self-hosted deployments. Key Features: Full-Agent Tracing: Captures every tool call, LLM hop, and branch so evals score complete trajectories. Research-Grounded Evals: Offers long-horizon uncertainty metrics and LLM-as-judge scoring to pinpoint where agents drift. Production Monitoring: Schedules eval runs on production traffic and alerts when metrics regress across versions. Agent-Native Workflow and Integrations: Ships a Skill file, CLI, and Python SDK that tie coding agents into major LLM providers. Pros Open source core: Apache 2. 0 licensing and self-hosting give teams control over data and deployment. Agent-focused workflow: The Skill and CLI let agents and CI scripts manage traces and evals. Low barrier to entry: A free Hobby cloud tier plus open source edition make starting instrumentation inexpensive. Cons Language focus: The main SDK targets Python, so teams in other languages must add custom wiring. Developer-centric setup: Effective use expects familiarity with tracing, agents, and CLI workflows, which can feel heavy for simple apps.