Agentic observability provides specialized observability for AI agents and multi-step workflows through dedicated dashboards, metrics, and trace visualization. Unlike traditional monitoring, which tracks single-shot inferences, agentic observability captures the complete lifecycle of autonomous-agent behavior—from initial reasoning through tool execution and final response.Documentation Index
Fetch the complete documentation index at: https://handbook.fiddler.ai/llms.txt
Use this file to discover all available pages before exploring further.
Dashboards & Visualization
Agentic observability uses a dedicated UI with Projects → Applications (instead of the legacy Projects → Models structure) designed specifically for observing agent workflows.Agentic Dashboards
Access pre-built dashboards optimized for agentic workflows:- Agent Performance Overview - Monitor success rates, latency, and throughput across all agents
- Workflow Execution Traces - Visualize complete multi-step reasoning chains from start to finish
- Tool Usage Analytics - Track which external tools and APIs your agents are calling
- Error & Exception Tracking - Identify where agent workflows fail and why
Trace Visualization
Every agent interaction is captured as a hierarchical trace showing:- Agent Steps - Each decision point in the agent’s reasoning process
- LLM Calls - All language model interactions with inputs, outputs, and metadata
- Tool Invocations - External function calls, API requests, and data retrievals
- Timing Information - Duration of each step to identify performance bottlenecks
- Parent-Child Relationships - How multi-agent systems coordinate and delegate tasks
- Navigate to your Application in the Fiddler UI
- Select a conversation or workflow from the list
- Click on any trace to expand the full execution tree
- Drill down into individual spans to see inputs, outputs, and metadata
Custom Dashboards
Create custom dashboards to monitor specific agent behaviors:- Combine multiple charts to track KPIs relevant to your use case
- Filter by agent type, user segments, or time periods
- Share dashboards with team members
- Set up alerts based on dashboard metrics
Metrics & Analytics
Agentic observability provides specialized metrics that go beyond traditional model monitoring:Agent-Specific Metrics
- Agent Success Rate - Percentage of workflows that complete successfully
- Tool Call Distribution - Which tools agents use most frequently
- Reasoning Chain Length - Average number of steps per workflow
- Agent Handoffs - How often agents delegate to other agents
- Retry & Recovery Rate - How often agents recover from errors
Performance Metrics
- End-to-End Latency - Total time from user request to final response
- Per-Step Latency - Duration of individual reasoning steps, LLM calls, and tool invocations
- Token Usage - Track LLM consumption across all agent interactions
- API Call Volume - Monitor external tool and API usage
Quality Metrics
- Response Accuracy - Validate agent outputs against expected results (requires ground truth)
- Hallucination Detection - Identify when agents generate unsupported claims
- Safety & Guardrails - Track safety violations and guardrail activations
- User Satisfaction - Capture feedback signals from end users
Analyzing Metrics
All metrics are available in:- Real-time Dashboards - Monitor live agent performance
- Historical Trends - Analyze patterns over days, weeks, or months
- Comparative Analysis - Compare different agent versions or configurations
- Custom Queries - Use Fiddler Query Language (FQL) for advanced analysis
Integration Options
Agentic observability ingests OpenTelemetry spans and traces from your agent applications. Choose the integration method that fits your framework:LangGraph Applications
Best for: Applications built with LangGraph or LangChain The Fiddler LangGraph SDK provides automatic instrumentation with zero code changes required.Strands Agent Framework
Best for: Applications built with Strands Agents The Fiddler Strands SDK integrates directly with the Strands framework for seamless monitoring.Custom Instrumentation (OpenTelemetry)
Best for: Custom agent frameworks or non-Python applications Use OpenTelemetry directly for maximum flexibility and control.Integration Comparison
| Feature | LangGraph SDK | Strands SDK | OpenTelemetry |
|---|---|---|---|
| Automatic Instrumentation | ✅ Zero code changes | ✅ Native integration | ❌ Manual setup |
| Framework Support | LangGraph, LangChain | Strands Agents | Any framework |
| Language Support | Python | Python | Python, Java, Go, JS, etc. |
| Setup Time | ~10 minutes | ~10 minutes | ~15 minutes |
| Customization | Medium | Medium | High |
Next Steps
- Getting Started: Learn the concepts behind agentic observability in our Agentic Observability Getting Started Guide
- Choose Integration: Pick your framework and follow the quick start guide above
- Explore Features: Set up dashboards, configure alerts, and analyze agent performance
- Advanced Topics: Learn about custom metrics, alerts, and segmentation