Respan

Route, track, and debug all LLM traffic.

What is Respan? Respan focuses on LLM engineering, giving teams a central gateway and tracing layer for AI applications. It routes traffic to providers such as OpenAI, Anthropic, Google Gemini, AI21 Labs, and AssemblyAI, then tracks tokens, costs, latency, and error rates in a single view. By pairing gateway-based logging with an OpenTelemetry tracing SDK, it lets engineers inspect entire agent workflows, from high-level tasks down to individual model calls. Key Features: Unified LLM gateway: Route requests through one base URL while still choosing models across multiple AI providers and tools. Token, cost, and latency analytics: Dashboard views show token usage, per-request cost, latency distributions, and error rates across all calls. Tracing SDK with decorators: OpenTelemetry-based SDK for Python and JavaScript uses decorators such as @workflow and @task to capture end-to-end traces, auto-attaching LLM calls. Rich attribution metadata: Attributes like customer_identifier, trace_group_identifier, and custom metadata help teams slice metrics by user, project, experiment, or environment. Flexible logging modes: Teams can either proxy traffic through the gateway by switching the base URL or log requests asynchronously via a dedicated logging endpoint. Pros Strong LLM observability: Fine-grained analytics make it much easier to understand where tokens, time, and errors are going.