Two of the most data-rich reports on AI agent engineering ever published landed within weeks of each other — and together they tell a story the industry has been waiting to hear. LangChain’s State of Agent Engineering, surveying 1,340 practitioners between November and December 2025, and Datadog’s State of AI Engineering, analyzing telemetry from over 1,000 production customers, converge on a decisive verdict: the era of agent experimentation is over. Organizations are shipping — but they are bumping into a new set of hard problems.
“Organizations are no longer asking whether to build agents, but rather how to deploy them reliably, efficiently, and at scale.” — LangChain State of Agent Engineering Report
The Big Number: 57% Have Agents in Production
LangChain’s survey reveals that 57% of organizations now have AI agents in production, up from 51% the previous year. Among large enterprises (10,000+ employees), that figure jumps to 67%. A complementary DigitalOcean survey of developers confirms this enterprise momentum while revealing a persistent gap between AI tool availability and team readiness. This isn’t a distant trend — it’s happening inside the companies that build your software, manage your finances, and treat your healthcare.
Datadog’s production telemetry confirms the explosion in a different way: LLM agent framework adoption nearly doubled year-over-year, from just over 9% in early 2025 to roughly 18% in early 2026. The number of services using agentic frameworks more than doubled in the same period.
| Metric | LangChain Survey | Datadog Telemetry |
|---|---|---|
| Production adoption | 57% overall, 67% at 10k+ orgs | Framework adoption ~18% of services |
| Sample | 1,340 practitioners | 1,000+ customers (billions of spans) |
| Top barrier | Quality (32%) | Rate limit errors (~60% of failures) |
Quality Is Still the #1 Barrier — But Security Is Rising Fast
For the second year running, quality — accuracy, relevance, consistency, and tone — is the top barrier to production deployment, cited by 32% of respondents. But the report uncovers a nuance: at organizations with over 2,000 employees, security surges to the #2 position (24.9%), overtaking latency. This shift mirrors findings from HBR’s research on AI agents and organizational design, which shows that governance and structural challenges intensify as agent deployments scale across the enterprise.
“For orgs with 10k+ employees, write-in responses pointed to hallucinations and consistency of outputs as the biggest challenge in ensuring agent quality.” — LangChain Report
Meanwhile, cost concerns have dropped significantly compared to prior years. Falling model prices and improved efficiency mean teams are less worried about the bill and more focused on whether the agent actually works correctly.
Datadog adds a crucial operational dimension: rate limit errors are the #1 cause of LLM call failures. In February 2026, 5% of all LLM call spans returned an error — and 60% of those were rate limit errors. Long-lived agent loops, ReAct-style reasoning chains, and multi-agent collaboration all exacerbate concurrency spikes. The infrastructure layer is becoming a real bottleneck.
Observability Is Now Table Stakes
If there is one unambiguous signal from both reports, it’s this: you cannot run agents in production without observability.
LangChain found that 89% of practitioners have implemented observability, rising to 94% among teams with agents already in production. Full per-step tracing is used by 62% overall and 71.5% of production teams.
“Without visibility into how an agent reasons and acts, teams can’t reliably debug failures, optimize performance, or build trust.” — LangChain Report
Datadog’s Guillermo Rauch (CEO of Vercel) put it even more bluntly:
“The next wave of agent failures won’t be about what agents can’t do. It’ll be about what teams can’t observe. Agents need the same production feedback loops we’ve always expected from great software.”
Yet evaluation remains a weak spot. Only 52.4% run offline evals on test sets, and just 37.3% run online evals — though that rises to 44.8% for production teams. Less than a third combine both approaches. Human review (59.8%) and LLM-as-judge (53.3%) dominate as evaluation methods, while traditional ML metrics like ROUGE and BLEU see low adoption.
The Multi-Model Reality
Both reports converge on a defining trend: multi-model strategies are the new normal.
- >70% of organizations use 3+ models (Datadog)
- >75% use multiple models in production or development (LangChain)
- Share using 6+ models nearly doubled year-over-year (Datadog)
- OpenAI GPT still leads (63% share), but Gemini gained 20 points and Claude gained 23 points YoY (Datadog)
However, Datadog flags a hidden cost: LLM tech debt. Teams rapidly adopt new models (Claude Sonnet 4.6 hit 17% adoption in its first month) but are slow to retire old ones. Older models like GPT-4o (22%) and Sonnet 4.5 (19%) persist at the same levels as newer releases. Each overlapping model adds operational overhead, evaluation burden, and provider-deprecation risk.
“In 2026, there appears to be no clear winner for model choice, and teams are increasingly keeping multiple models in flight.” — Datadog Report
The Prompt Engineering Challenge: System Prompts Are Eating the Context Window
One of Datadog’s most striking findings: 69% of all input tokens in traced LLM calls are system prompts — the scaffolding of instructions, policies, and tool definitions that make agents work. Yet only 28% of LLM calls on caching-capable models show any cached-read input tokens. The majority re-process the full prompt every time.
Meanwhile, token usage per request more than doubled for median customers and quadrupled for power users at the 90th percentile year-over-year. Context windows of up to 2 million tokens mean context capacity is no longer the bottleneck — context quality is.
“Organizations that invest in context engineering — retrieval quality, summarization, deduplication, clear information hierarchy — will close the gap between what long-context models allow and what production agents can reliably work with.” — Datadog Report
What Teams Are Actually Building
LangChain’s survey reveals the top three agent use cases:
- Customer service — 26.5% (growing rapidly, suggesting organizations are comfortable putting agents in front of customers)
- Research & data analysis — 24.4%
- Internal workflow automation — 18%
At large enterprises (10k+), the priorities shift: internal productivity ranks first (26.8%), followed by customer service (24.7%) and research (22.2%).
The most-used daily agents, by write-in response, are coding assistants (Claude Code, Cursor, GitHub Copilot, Amazon Q, Windsurf), research agents (ChatGPT, Claude, Gemini, Perplexity), and custom agents built on LangChain/LangGraph for QA, search, SQL generation, demand planning, and workflow automation.
“A meaningful minority noted they don’t yet use agents beyond LLM chat or coding assistance — broader ‘agentic everything’ is still in its early innings.” — LangChain Report
Yet some organizations are already far past the experimentation phase. SAP’s Autonomous Enterprise initiative now runs over 200 agents in production, offering a glimpse of what scaled, industrial-grade agent deployment looks like.
Self-Hosted Models and Fine-Tuning: The Myths
Both reports puncture two common narratives:
- Self-hosted models: About 1/3 of organizations invest in infrastructure to run their own models, driven by cost, data sovereignty, and regulation — but this is a minority strategy.
- Fine-tuning: 57% are not fine-tuning at all. The dominant approach remains base models + prompt engineering + RAG. Fine-tuning is reserved for narrow, high-volume use cases where the cost-benefit is clear.
The Bottom Line
Taken together, these two reports paint a picture of an industry in transition. AI agents are no longer a science experiment — they are being deployed in customer-facing roles, embedded in enterprise workflows, and managed with the same rigor (and the same headaches) as any production software system.
The key takeaways for anyone building agents in 2026:
- Invest in observability first — without it, you are flying blind
- Plan for multi-model from day one — build routing and fallback into your architecture
- Treat quality as a continuous process — evaluation must be offline, online, and automated
- Manage your context like code — system prompts are the new codebase
- Rate limits will bite you — design for graceful degradation and backpressure
The floor has lifted. The ceiling hasn’t been found yet.
Sources: LangChain State of Agent Engineering, Datadog State of AI Engineering