AgentSight is an eBPF-based observability tool that captures LLM API traffic, prompts, tool calls, and latency data directly from the Linux kernel—requiring zero code changes, zero SDKs, and zero framework lock-in. As AI agents move from demos to production, this kernel-level approach to AI agent observability challenges the SDK-instrumentation model used by LangSmith, LangFuse, and OpenTelemetry. This analysis breaks down the architecture, the trade-offs, and where it fits in a production stack. You can review the full launch context in the AgentSight Hacker News thread.
The Observability Gap in Production AI Agents
AI agents are non-deterministic, multi-step systems that chain LLM calls, tool invocations, and external API requests. Traditional APM tools (Datadog, New Relic) see generic HTTP spans but not the semantic payload—the actual prompt, completion, token counts, or which agent step failed. The current generation of agent tracing tools fills this gap via SDKs, which introduces three structural problems:
- Instrumentation friction: Every service must import and configure a tracing library; coverage gaps appear the moment a developer forgets or a dependency bypasses the wrapper.
- Framework coupling: LangSmith and LangFuse trace through framework hooks; agents built on raw OpenAI/Anthropic SDKs, custom HTTP clients, or polyglot stacks (Python + Node + Go) slip through.
- Telemetry drift: The observed behavior of an SDK depends on framework version and wrapping order—your trace may not reflect what actually left the process.
AgentSight sidesteps all three by observing traffic at the point of truth: the kernel.
What AgentSight Does
AgentSight instruments the boundary between your agent processes and the outside world. Once deployed, it automatically discovers and records:
- LLM API calls: Full request/response payloads to OpenAI, Anthropic, and other providers, captured regardless of which SDK or HTTP client made them.
- Per-process attribution: Every call is mapped to its originating PID, container, and cgroup—so you know which agent, step, or service produced each request.
- Latency and token telemetry: Time-to-first-token, total round-trip time, and token usage extracted from response payloads.
- Tool and external API invocations: Egress traffic to search APIs, databases, and internal services appears alongside LLM calls, giving a complete execution trace of the agent loop.
- Framework-agnostic coverage: LangChain, AutoGen, CrewAI, raw provider SDKs, or fully custom agent loops—there is nothing to integrate.
How It Works: The eBPF Deep Dive
The core technical insight is that encrypted traffic must be plaintext inside the process before TLS encryption. AgentSight uses uprobes (user-space probes) attached to TLS library entry points—SSL_write and SSL_read in OpenSSL, BoringSSL, and equivalents—to capture request and response bodies before they hit the network stack, kernel-side.
Execution path
- Probe attachment: eBPF programs attach to TLS functions in every matching process; binary offsets are resolved automatically via symbol tables.
- In-kernel capture: Payloads are copied into BPF ring buffers; filtering happens in-kernel to minimize overhead, discarding irrelevant traffic before it reaches userspace.
- Process enrichment: cgroup, namespace, and container metadata are joined with each event, enabling per-agent and per-deployment attribution in Kubernetes environments.
- Userspace reassembly: An agent-side daemon reassembles chunked TLS writes into complete HTTP requests, parses JSON payloads, and extracts model, tokens, and tool-call structure before shipping to storage or a dashboard.
This is the same architectural pattern proven by projects like Pixie, Cilium's Hubble, and DeepFlow for general service observability—AgentSight applies it specifically to LLM traffic semantics. The result: near-zero marginal overhead on your agent processes (no application-thread blocking), and a capture layer that no application code path can bypass.
AgentSight vs. SDK-Based Tracing
- LangSmith / LangFuse: Rich span nesting and evaluation workflows, but require SDK adoption, add per-call overhead in application threads, and only trace what the wrapper sees.
- OpenTelemetry (GenAI semantic conventions): Vendor-neutral standard, but instrumentation is still manual or framework-dependent; multi-language agent stacks multiply integration effort.
- AgentSight: Universal capture with zero instrumentation, but kernel-level egress visibility—no awareness of internal agent state, planning steps, or in-memory reasoning that never produces I/O.
The pragmatic conclusion from early adopters: these are complementary, not competing. eBPF capture guarantees complete ground-truth I/O; SDK tracing adds structured semantic context. Teams running regulated or brownfield agent deployments get the most immediate value from AgentSight because retrofitting SDKs across an existing agent fleet is often impossible.
Limitations and Trade-offs
- Linux-only: eBPF is a Linux kernel technology; recent kernels with BTF support are required for portable CO-RE binaries.
- Privileged deployment: Loading BPF programs requires
CAP_BPF/root or elevated privileges—a genuine consideration for multi-tenant Kubernetes clusters and security review. - Plaintext capture risk: Capturing decrypted payloads at the kernel means prompts, completions, and any PII flowing through them are now logged. Redaction, encryption-at-rest, and retention policies are mandatory; this is the single biggest discussion point in the AgentSight launch discussion on Hacker News.
- No internal state: You see what crosses the process boundary—retries, routing logic, and in-memory planner state remain invisible without complementary instrumentation.
Who Should Adopt It
- Teams with brownfield agents: Production agents you cannot refactor to add tracing SDKs.
- Polyglot stacks: Agent logic split across Python orchestration, Node services, and Go tools.
- Security and audit teams: A tamper-resistant record of every model interaction, useful for compliance and incident forensics (e.g., detecting prompt-injection-driven exfiltration attempts).
- Platform teams: One deployment covers every current and future agent—new services inherit observability automatically.
Verdict: A Structural Advantage Worth Watching
AgentSight represents the correct long-term direction for AI agent observability: capture at the kernel, attribute at the platform layer, and never depend on developers remembering to instrument. The privacy and privileged-deployment caveats are real but manageable with standard data-governance practice. For teams operating agents where failure modes are opaque and debugging costs are high, the zero-instrumentation guarantee alone justifies evaluation. For deeper analysis on agent architecture and production LLM systems, browse our studio blog, or explore our AI development services to see how we build and instrument agent systems end-to-end.