What Makes AI "Agentic" — and Why It Changes Everything
For the past two years, enterprises have deployed AI in a reactive mode: user asks a question, LLM generates an answer, human decides what to do next. Useful. But fundamentally limited.
Agentic AI breaks this pattern. An AI agent doesn't just respond — it plans, acts, observes the result, and re-plans until a goal is achieved. It can call APIs, run code, search the web, write and execute queries, send emails, and orchestrate other agents — all autonomously, with minimal human intervention.
The shift from "AI as answer engine" to "AI as autonomous system" is the most significant architectural change since the move to cloud. The enterprise implications are enormous — and so are the risks if you get the architecture wrong.
This guide covers what we've learned from deploying 12+ agentic AI systems in production across insurance, legal, financial services, and logistics.
The Agentic AI Stack in 2026
Core Primitives
Every agentic system rests on three primitives:
1. The Planner (Reasoning Engine)
The LLM that decides what to do next. In 2026, GPT-4o, Claude 3.7 Sonnet, and Gemini 2.0 Pro are the dominant planners for enterprise use cases. The planner takes a goal and current state and outputs a next action.
2. The Tool Layer
The functions the agent can call: web search, database queries, API calls, code execution, file operations, sending notifications. Tool design is where most enterprise deployments fail — poorly scoped tools lead to agents that get stuck or hallucinate capabilities they don't have.
3. The Memory System
What the agent knows and remembers:
- In-context memory: what's in the current prompt window
- External memory: vector databases, key-value stores, document stores
- Episodic memory: logs of past runs, successes, and failures
Orchestration Frameworks
| Framework | Best For | Maturity |
|---|---|---|
| LangGraph | Complex multi-step workflows with conditional branching | Production-ready |
| CrewAI | Multi-agent collaboration with role assignment | Maturing fast |
| AutoGen | Research and prototyping, multi-agent conversations | Research-grade |
| Pydantic AI | Type-safe agentic workflows with strong validation | Emerging |
| Semantic Kernel | Microsoft ecosystem, .NET-first enterprises | Enterprise-ready |
We use LangGraph for the majority of our enterprise deployments. Its graph-based state machine model maps naturally to complex business processes with conditional logic and human-in-the-loop checkpoints.
Agentic Architectures: What Actually Works
Pattern 1: Single Agent with Tools (Most Common)
The simplest and most reliable architecture. One LLM with access to 3–8 well-scoped tools. The agent plans, uses tools, observes results, and reports back.
When to use: Well-defined tasks with limited tool space. Research assistants, document processors, data extractors, classification agents.
Real deployment: Our insurance claims routing agent uses this pattern — GPT-4o with 5 tools (PDF extractor, classifier, structured data writer, routing API, audit logger). Processes 2,000+ claims daily with 99.6% accuracy.
Goal: Process incoming claim
→ Tool: extract_pdf(document)
→ Tool: classify_claim(data)
→ Tool: extract_fields(claim, schema)
→ Tool: route_to_adjuster(claim_id, team_id)
→ Tool: log_audit_trail(claim_id, actions)
→ Done
Pattern 2: Supervisor + Specialist Agents
A planner agent decomposes a complex goal into subtasks and delegates to specialist agents. Each specialist has a narrow tool set optimised for its domain.
When to use: Complex multi-domain tasks, research pipelines, content generation workflows where different expertise is needed at different stages.
Real deployment: Our weekly market intelligence pipeline (Xenith Radar) uses a Supervisor agent that delegates to: a Hiring Signal Agent, a Funding Round Agent, a Competitive Intelligence Agent, and a Report Writing Agent. The Supervisor synthesises their outputs into a final market report.
Supervisor: "Produce weekly market report for AI Infrastructure sector"
→ Delegate to HiringAgent: scan LinkedIn signals for target companies
→ Delegate to FundingAgent: check Crunchbase and press for funding news
→ Delegate to CompAgent: monitor pricing pages, release notes, job postings
→ Delegate to ReportAgent: synthesise findings into structured report
→ Supervisor: review, quality-check, deliver to Slack
Pattern 3: Reflexion / Self-Correcting Agents
The agent generates an output, evaluates it against a rubric, and iterates until it meets the quality threshold — without human intervention.
When to use: Coding agents, document drafting, SQL generation, structured data extraction where quality is measurable.
Critical implementation note: Always cap iterations (max 3–5 loops). Uncapped reflexion loops are one of the most common causes of runaway API costs in production.
The Six Failure Modes of Agentic AI (and How to Prevent Them)
Most agentic AI projects fail not because the LLM is wrong, but because the orchestration architecture has unaddressed failure modes.
Failure Mode 1: Tool Call Loops
The agent calls Tool A, gets an unexpected result, decides to call Tool A again, gets the same result, calls it again — infinitely.
Fix: Implement a maximum tool call budget per agent run. We use 25 tool calls as a default ceiling, with an alert at 15. If the budget is hit, the agent escalates to human review.
Failure Mode 2: Context Window Overflow
For long-running agents, the accumulated history fills the context window. The LLM starts hallucinating or making random decisions because it can't "see" the beginning of the task.
Fix: Implement sliding window memory with summarisation. After every N tool calls, summarise the history into a compact representation and drop the raw history.
Failure Mode 3: Tool Hallucination
The agent "invents" tool calls for tools that don't exist, or calls real tools with incorrect parameters.
Fix: Use strict Pydantic schemas for all tool inputs. Validate before execution. Use function calling (not text parsing) for all tool invocations — structured outputs from OpenAI, Anthropic, or Gemini reduce hallucination by ~60% versus prompt-parsed tool calls.
Failure Mode 4: Runaway Costs
An agent that runs for 3 hours without human checkpoints can consume hundreds of dollars in API calls. In one case we reviewed externally, a poorly implemented research agent cost $840 in a single overnight run.
Fix: Implement per-run cost ceilings with automatic pause. We use LangSmith to track token consumption in real time and trigger a Slack alert + automatic pause if a single run exceeds a cost threshold.
Failure Mode 5: Irreversible Actions Without Confirmation
An agent with write permissions that takes an action it can't undo — deleting records, sending emails, triggering payments.
Fix: Classify all tools as read-only or write-capable. All write-capable tools should require explicit human confirmation in the workflow, or be preceded by a plan-approval step where the agent presents its intended actions before executing.
Failure Mode 6: Prompt Injection via Tool Outputs
If your agent reads web pages, documents, or emails and passes that content into the next LLM call, an attacker can embed instructions in that content to hijack the agent's behaviour.
Fix: Sanitise all tool outputs before they re-enter the prompt. Use a secondary "safety filter" LLM call to detect adversarial content in retrieved documents before the main agent processes them.
Human-in-the-Loop: Where to Add It
Fully autonomous agents are powerful but require significant trust. For enterprise deployments, we recommend a tiered HITL model:
| Risk Level | Example Actions | HITL Requirement |
|---|---|---|
| Read-only | Search, classify, extract | No checkpoint required |
| Low-write | Log entries, status updates | Checkpoint after batch of 50+ |
| Medium-write | Send notifications, update records | Checkpoint on each action |
| High-write | Delete data, send external emails, trigger payments | Always require human approval |
LangGraph's interrupt mechanism makes this clean: pause execution at any node, surface the agent's proposed action to a human review interface, and resume or redirect based on their decision.
Evaluating Agentic Systems: The Evals Framework
This is where most enterprise teams fail. They deploy an agent, it "seems to work", and they call it done. Six months later, the model drifts, edge cases accumulate, and production quality degrades.
The evals you need before launching:
1. Task Success Rate
Does the agent achieve the goal? Measure on a held-out dataset of real production inputs. For our claims routing agent: did the claim end up in the right team's queue?
2. Tool Call Efficiency
How many tool calls does the agent use per task? Fewer is generally better — high tool call counts indicate planning inefficiency. Track mean and p95.
3. Hallucination Rate
On tool outputs where there's a ground truth, how often does the agent fabricate information? Use a structured extraction + comparison approach.
4. Latency and Cost per Task
Track p50, p95, p99 latency and dollar cost per agent run. Set regression thresholds.
5. Human Escalation Rate
What percentage of tasks does the agent fail to complete autonomously and escalate to a human? Healthy range for most enterprise agents: 2–8%. Above 15% means the agent's capability is mismatched to its task definition.
Choosing Your LLM for Agentic Workloads
Not all LLMs are equal for agentic tasks. Our 2026 production benchmarks:
| LLM | Planning Quality | Tool Call Accuracy | Speed | Cost |
|---|---|---|---|---|
| GPT-4o | ★★★★★ | ★★★★★ | ★★★★ | ★★★ |
| Claude 3.7 Sonnet | ★★★★★ | ★★★★★ | ★★★★ | ★★★ |
| Gemini 2.0 Pro | ★★★★ | ★★★★ | ★★★★★ | ★★★★ |
| Llama 3.3 70B (private) | ★★★★ | ★★★★ | ★★★★ | ★★★★★ |
| Mistral Large | ★★★ | ★★★★ | ★★★★★ | ★★★★★ |
For most enterprise agents: Claude 3.7 Sonnet or GPT-4o as primary planner, with a cheaper/faster model (Gemini Flash, GPT-4o-mini) for simple classification sub-tasks.
Private/on-premise: Llama 3.3 70B via Ollama or vLLM gives 80–85% of frontier model quality at zero variable cost — the right choice for regulated industries with data sovereignty requirements.
Production Checklist Before You Launch
Before any agentic system goes live in enterprise production, verify:
Architecture
- Maximum tool call budget implemented (25 calls default)
- Context window overflow handling (sliding window or summarisation)
- All tool inputs validated with Pydantic schemas
- Write-capable tools have human confirmation gates
- Prompt injection sanitisation on all retrieved content
Cost & Observability
- Per-run cost ceiling with Slack alert
- LangSmith or equivalent tracing on all agent runs
- Token consumption visible in real time
- Cost per task tracked in your metrics system
Evals & Quality
- Evals baseline established on 50+ real production inputs
- Human escalation rate < 15%
- Tool call efficiency within acceptable range
- Regression tests that run on every deploy
Security & Compliance
- Agent actions logged to immutable audit trail
- No PII passed to external APIs without consent
- Role-based access for HITL review interface
- Incident response runbook written and tested
What Agentic AI Cannot Do (Yet)
Honesty matters. Agentic AI in 2026 is genuinely powerful, but there are hard limits:
Long-horizon planning without feedback: Agents struggle with tasks that require 50+ steps without any intermediate validation. They accumulate errors. Keep tasks to 10–15 steps where possible.
Reliable arithmetic: LLMs still make arithmetic mistakes. For any financial calculation, always route through code execution (Python, not LLM reasoning).
Novel creative judgment: Agents are excellent at process-following and synthesis, but poor at genuine novelty. A legal contract review agent will find standard clauses; it won't invent a new legal strategy.
Handling ambiguity gracefully: Agents tend to make assumptions rather than ask for clarification. Design your systems to surface ambiguity early — either via a clarification step in the workflow, or via a human review of the agent's plan before execution.
Getting Started: Our Recommended Path
For most enterprises new to agentic AI, we recommend this sequence:
Week 1–2: Identify the right first use case
The best first agentic system is one where: (a) the task is currently done by a human, (b) the steps are predictable and documentable, (c) the inputs and outputs are digital, and (d) errors are non-catastrophic and detectable.
Week 3–4: Build a scoped POC
Single agent, 3–5 tools, one task. Measure task success rate manually. Identify where it fails. Don't skip this step.
Week 5–8: Harden for production
Add the production checklist items above. Build the evals harness. Add monitoring. Get human review working.
Week 8+: Scale
Add more tasks, more tools, sub-agent delegation. Use the evals framework to detect regressions as you grow.
The Bottom Line
Agentic AI is not hype. The enterprises that deploy well-architected agentic systems in 2026 will have material productivity and cost advantages within 12 months.
The enterprises that deploy carelessly — no tool budgets, no evals, no HITL design — will have expensive failures and spend 2027 cleaning up.
The architecture isn't the hard part. The hard part is the operational maturity: monitoring, evals, incident response, and a clear mental model of what the agent can and cannot do. Build that first.
Building an agentic AI system and want an experienced pair of eyes on the architecture? Book a free technical review — we'll spend 30 minutes on your specific design and tell you honestly where the risks are.