I’ve been looking for a pragmatic way to put product analytics where my teams already work—inside Slack and Microsoft Teams. The moment insights are one message away, cycle time shrinks, debates get crisper, and experiments move faster. That’s why I’m bringing Amplitude Global Agent into our daily decision flow to deliver instant, source-backed answers with visual clarity and actionable next steps.
Connect Amplitude Global Agent to Slack or Microsoft Teams to answer questions with source-backed analytics, charts, and recommended actions like A/B tests.
What excites me most is the shift from dashboards to dialogue. Instead of digging through reports, I can ask a focused question in Slack—“How did activation change week-over-week for our self-serve cohort?”—and get a chart in-channel, complete with recommendations that point me toward the next best move. This is Agent Analytics done right: faster insight loops, reduced context switching, and more confidence in the decisions we make every day.
From a product management perspective, this integration strengthens continuous discovery and aligns product trios around the same truth. Engineers, designers, and PMs see the same chart, discuss trade-offs in the same thread, and can agree on an action—often an A/B test—within minutes. It’s a lightweight but powerful way to support product-led growth and keep our roadmap tied to measurable outcomes.
In practice, the questions I ask the most look like this: “Which onboarding step causes the biggest drop-off this month?”, “Which channels drive the highest L28 activation rate?”, and “Where did retention improve after our pricing change?” In each case, the Agent returns charts we can share instantly with stakeholders, plus recommended actions like A/B test ideas to validate hypotheses quickly. The result is a reliable rhythm: ask, see, align, act.
Governance matters just as much as speed. We’re configuring strict permissions, role-based access, and purposeful channel placement so analytics land where they should—no broader, no narrower. We’re also leaning into clear query prompts and naming conventions for events and properties to help the Agent retrieve precisely what’s needed, every time. The aim is a high-signal, low-noise system that maintains trust while accelerating decisions.
To embed this into our operating cadence, I plug the Agent into three moments: daily standups (to scan activation, conversion, and incidents), weekly product reviews (to align on experiment status and next bets), and executive QBR prep (to pull clean, shareable charts fast). Because the insights arrive in Slack or Microsoft Teams, our conversations stay focused and traceable, and decisions get documented in the same place they were discussed.
We’ll measure impact with simple, telltale indicators: fewer ad-hoc analytics requests, faster time from question to decision, increased A/B test velocity, and clearer links between recommended actions and outcome metrics like activation and retention. My bar is straightforward—if this Agent can help one team make a better decision per day, it will more than pay for itself across the org.
If you’re considering a similar move, start small: connect one high-signal channel, curate a handful of common queries, and coach your team on good prompts. Within a week, you’ll feel the difference. When analytics become conversational, momentum follows—and your product strategy benefits from sharper, faster, and more transparent decision-making.
Inspired by this post on Amplitude – Best Practices.
Shipping agentic AI into production is exhilarating—until a flaky output torpedoes trust. Over the past year, I’ve led teams at HighLevel to operationalize agents across customer-facing and internal workflows, and I’ve learned that reliability isn’t an afterthought; it’s an architecture. In this piece, I share the AI Agent Orchestration Patterns for Reliable Products that consistently deliver dependable outcomes at scale.
When we talk about orchestration, we’re talking about more than a single prompt. The shift is from monolithic calls to coordinated “agentic AI” where routers, planners, and specialists collaborate through structured “AI workflows.” In practice, I rely on a few canonical patterns: a planner–executor loop for multi-step tasks, a router–specialist setup for skill selection, and a “retrieval-first pipeline” that grounds generation with authoritative context before a single token is produced.
Reliability-by-design starts with typed inputs/outputs and strict validation. I standardize on JSON schemas, enforce tool/function signatures, and implement idempotency keys so retries don’t wreak havoc on downstream systems. Timeouts, circuit breakers, and backpressure protect the platform under load, while rate limiting and dead-letter queues keep failure modes contained. Most importantly, we engineer graceful degradation: agents “abstain” when uncertain, fall back to deterministic paths, and escalate to humans instead of guessing.
Safety is a first-class concern, not a bolt-on. Our “AI risk management” pipeline includes PII redaction, allow/deny lists for tools and data, and the principle of least privilege for every connector (yes, even the ChatGPT connector). We codify policy-as-code for repeatability and require human-in-the-loop approvals for sensitive or irreversible actions. In my experience, clear red lines and reversible defaults prevent the vast majority of regrettable outcomes.
Without strong “observability,” you’re flying blind. I instrument agents with an “Agent Analytics” layer that captures traces, spans, tool invocations, and token usage across the entire chain. The essential metrics are outcome quality (task success rate), latency (p50/p95), tool failure rates, cost per task, and user-level satisfaction signals. Cross-agent lineage allows us to pinpoint where a plan went awry and which tool or prompt introduced drift—vital for rapid remediation.
Quality improves fastest when it is measured relentlessly. I practice “eval-driven development” with golden datasets, rubric-based scoring, and risk-weighted sampling of edge cases. LLM-as-judge can help, but we always calibrate against human ratings and monitor agreement. In production, I blend online metrics with controlled “A/B testing” and plan experiments to hit a realistic minimum detectable effect (MDE). The result is a virtuous loop where prompt tweaks, tool changes, and retrieval adjustments are verified before wide rollout.
Agents need the same rigor we expect from any modern system. I gate releases through “CI/CD” with linting for prompts, schema checks for tools, and simulation runs for critical paths. “Feature flags” enable shadow and canary deployments so we can throttle exposure by segment or workflow. I also track reliability with “DORA metrics” and “deployment frequency,” and I partner closely with “SRE” for on-call coverage, runbooks, and incident postmortems tailored to agent failure modes.
Context is a resource to allocate, not a bottomless pit. Thoughtful “context window management” means curating retrieval, summarizing long-running threads, setting memory time-to-live, and constraining what the agent can see at any given step. I bias hard toward retrieval over recall, keep chunks small and semantically precise, and validate that the “retrieval-first pipeline” truly returns the right evidence—not just the nearest match.
In day-to-day product work, I lean on a compact playbook: a router that selects the best specialist; a planner that decomposes tasks and allocates tools; a deterministic guard that verifies preconditions; an execution loop with explicit budgets; and a fallback policy that prefers abstaining over hallucinating. Together, these patterns create an agent that behaves like a dependable teammate rather than a creative wildcard.
No architecture thrives without the right rituals. Product trios keep discovery continuous, while clear outcomes (not output) align teams on value instead of vanity. We map risks early, maintain a public quality dashboard, and rehearse failure recoveries so incidents never become improvisations. The cultural signal is simple: we celebrate root-cause clarity and safe iteration over heroics.
If you’re just starting, implement three patterns first: retrieval before generation, abstain-and-escalate for low confidence, and canary releases under feature flags. Instrument everything from day one, run a weekly eval review, and expand scope only when the data says you’re ready. With these habits, your agents will earn user trust—and keep it.
To truly transform with AI, I’ve learned it’s never just about the technology—it’s about redesigning how we work. The teams that win don’t bolt AI on; they re-architect around it. That means rethinking roles, workflows, and governance to build a system that sustains and improves AI performance over time.
In The 2026 Customer Service Transformation Report, teams at every stage of maturity describe human agents taking on more proactive work—training AI systems, handling the hardest queries, and owning tasks that demand judgment. Job descriptions are shifting, too, with many organizations explicitly adding AI-related responsibilities.
I’m also seeing a clear rise in dedicated AI specialists. Conversation analysts, knowledge managers, and AI operations leads are fast becoming standard. For support professionals, this opens new, higher-leverage career paths—and creates a talent pipeline that blends service excellence, data fluency, and product thinking.
Support once centered on queue-level activity—ticket triage, routing, translations, and answering FAQs. Now, as AI handles more frontline interactions, our human roles are moving up the stack toward optimization, oversight, and continuous improvement.
According to the latest research, 45% of teams report updating job descriptions to include AI-related responsibilities, with 40% saying their human agents are now more focused on training AI systems. Another 27% report that human agents primarily handle the most complex escalations and edge cases, while a quarter say agents are doing more consultative and strategic work.
Even at the initial deployment stage, 16% of teams report spending less time handling support volume since implementing AI – and among teams who’ve reached maturity, that figure rises to 28%.
When Intercom’s Research, Analytics & Data Science (RAD) team interviewed 166 of our customers, similar themes emerged. Nearly all participants (≈95%) reported meaningful workflow changes, with manual processes being handled by AI, and humans focusing more on monitoring or fine-tuning AI outputs. Eighty-three percent of participants also reported seeing their team’s roles and responsibilities change to become more strategic and supervisory in nature.
AI is reshaping support teams: organizations are adding conversation analysts (32%), knowledge managers (30%), AI operations leads (28%), and support automation specialists (24%). Just 8% report no new AI roles.
It’s not just the work that’s evolving; organizational structures are, too. Some teams are reallocating existing talent into AI-focused roles; others are hiring entirely new skill sets. Many of the most common job titles in this space didn’t exist two years ago.
Consider a Senior AI Knowledge Manager, Beth-Ann Sher, who transitioned from a help center manager role. Like many careers transformed by AI, her work evolved from administrative to strategic. Instead of focusing solely on customer-facing, self-serve content, her mandate expanded to designing and optimizing knowledge inputs that directly improve AI Agent Fin’s performance—work that materially lifts resolution rates.
Or look at a Senior Conversation Designer, Fred Walton, hired specifically for an AI-first function. He focuses on frictionless customer journeys with Fin, smoothing handoffs between automation and human support while keeping customer satisfaction front and center—hallmarks of mature AI workflows and conversation design.
In high-performing organizations, roles like these typically sit within a dedicated AI support team under senior CS leadership. Clear ownership and accountability for AI performance is critical; without it, optimization stalls and trust erodes.
These shifts aren’t isolated. Take Robb Clarke from RB2B. He went from Head of Technical Operations to Head of AI. With Fin, his focus moved from repetitive support questions to managing knowledge and improving the system behind it—freeing him to be proactive about product improvements and fix issues before they hit customers.
Or consider Eric Broulette from Bloomerang, a support leader who leaned into AI and became the VP of Support and Education. By deploying Fin, his team found breathing room to invest in what’s next. Agents stepped into new roles, contributed to meaningful projects, and built skills that had previously felt out of reach. As Eric puts it: “Do not wait to embrace AI. It will unlock more career growth for your teams than you can imagine.”
Leaders are racing ahead with real AI in support. Explore the 2026 Customer Service Transformation Report to see where deployment is stalling, benchmark your team, and get practical steps to scale automation that delights.
Bringing AI into support will eventually change every agent’s day-to-day work. For leaders at the start of the journey, that can feel daunting. My perspective: the most successful teams treat this as an operating model shift, not a tooling rollout—anchored in AI Strategy, governance, and continuous improvement.
Be transparent about what’s changing, why it matters, and how success will be measured. Define how AI performance will be evaluated (resolution rate, containment, CSAT impact), empower agents to train and improve the system, and communicate how responsibilities will evolve. When teams help build the AI, they’re invested in making it great.
Here’s the playbook I rely on with support leaders: First, reset expectations about time allocation—less time in the queue, more time improving the AI system that serves the queue. Second, elevate knowledge management as a core capability. Prioritize content quality and coverage for your AI Agent, and carve out dedicated “out of the inbox” time so every agent contributes. Third, keep outcome metrics—especially resolution rate—front and center. It gives the team a north star for experimentation and iteration.
Scaling AI is as much a people challenge as it is a technology challenge. As automation takes on more work, support roles become more proactive, strategic, and cross-functional—even early in the journey. Responsibilities expand, new roles emerge, and team structures adapt to concentrate on and amplify AI performance. In the process, support careers are transformed.
If you’re leading this shift, now’s the moment to reimagine your operating model: clarify ownership, invest in knowledge and conversation design, adopt eval-driven development, and build the muscle for continuous improvement. That’s how you move from tickets to strategy—and unlock compounding value for your customers, your business, and your teams.
Today, I’m excited to share 12 major updates to Fin’s Procedures and Simulations—the foundation that lets Fin handle complex work while keeping teams fully in control of the customer experience.
In my work building AI workflows with product and support leaders, I’ve seen how the right blend of natural language instructions, deterministic controls, and fully agentic behavior turns Fin into a reliable problem solver. Procedures make this blend possible by enabling Fin to act like a human—yet with the repeatability and governance of software. Simulations then let us test those complex Procedures at scale before they reach customers, so we can deploy with confidence.
Together, these capabilities make Fin self-manageable, transparent, and ready for genuinely complex work.
Here’s what’s new at a glance: we’ve made Procedures easier to build and maintain; enhanced deterministic controls for precision and policy compliance; expanded agentic behavior so Fin can adapt in real time; and delivered more powerful Simulations to validate end-to-end workflows before go-live.
Why did we build this? Many teams see early AI gains in speed, coverage, and cost to serve—but then hit a ceiling. They keep AI confined to simple automation and information retrieval, rather than setting it up to handle the nuanced, multi-step workflows they still trust to humans. We designed Procedures and Simulations to remove that ceiling, so teams can confidently set up, govern, and iterate on complex AI workflows without bottlenecks.
Follow the AI lifecycle as it cycles from Analyze to Train to Test to Deploy. This streamlined loop spotlights the TRAIN phase, underscoring faster iteration and feedback that power more capable procedures and realistic simulations.
We also heard that teams needed an easy way to connect data so Fin could reliably check customer status or eligibility and then take action. And they didn’t want to route through engineering every time they needed to create or amend logic for mid-conversation decisions. Procedures combines natural language instructions and intuitive data connector setups. You tell Fin in your own words how you want it to behave, and you’ll be guided through creating conditional steps so Fin will react consistently, with the option to add in any code snippets for circumstances where absolute precision is required. Once you build one Procedure, we believe you’ll want to build several, so Fin will constantly read the conversation it’s in to ensure it’s following the most relevant Procedure, and jump to a more relevant one if the user intent changes.
I know that taking something like this live the first time can feel like a leap of faith. That’s exactly why we built Simulations—to test Procedures comprehensively, uncover edge cases, and launch with confidence.
Reaching mature deployment takes a deliberate, ongoing commitment to training workflows, validating them before deployment, measuring performance in production, and refining them over time. At Intercom, we call this the Fin Flywheel: train, test, deploy, analyze. Procedures form the foundation of the train stage, and Simulations make the test stage reliable at scale. Together, they enable Fin to handle complex work, and teams to stay in control of it.
Procedures: Define exactly how Fin handles complex work. With Procedures, I can set Fin up to resolve complex, time-consuming queries that require multiple steps or business logic. Fin follows standard operating procedures and applies sound judgment—just like a seasoned teammate—so even complicated queries are resolved in controllable, predictable ways.
A snapshot of the Procedures builder in action, mapping a clear path for handling damaged food orders while letting teams train Fin on examples, target channels, quickly test updates, and publish with Set live.
Procedures combine three powerful elements. First, natural language instructions. You write a Procedure in plain language, just like documenting a process for a new teammate. You can paste in your existing SOPs, write from scratch, or let AI draft them for you, then iterate yourself.
What’s new: Draft Procedures with AI. Share an outline of your process and Fin drafts a complete Procedure using your conversation history, knowledge hub content, and relevant data. If additional context is needed, it prompts you with clarifying questions to make sure the Procedure is thorough and tailored to your use case, significantly reducing setup time. For example: if you’re creating a refund workflow, the system can draft conditional paths for eligibility, approval thresholds, and verification steps based on your historical cases and policies.
What’s new: Break complex workflows into Sub-procedures. Write a process once and reference it across multiple Procedures by breaking it down into reusable steps, called Sub-procedures. This makes workflows easier to read, faster to build, and simpler to maintain as things change.
Second, deterministic controls. Natural language is flexible, but some steps need to be exact. You can layer in deterministic controls where precision matters, starting with a fully natural language Procedure and introducing structure gradually where it adds value: conditional steps (branching logic) to handle decision points so Fin’s behavior is consistent and predictable; data connectors so Fin can pull information from your tools or take actions automatically; code snippets for when absolute accuracy is essential; and checkpoints to pause for approval or hand off to a teammate.
Fin demonstrates structured troubleshooting: a transaction dispute flow with eligibility checks, clear IF/ELSE steps, and quick Data Connector actions like freezing a card or pulling invoices, streamlining complex support tasks.
What’s new: Instruct Fin to read specific content from your knowledge hub. You can set clear rules for Fin to reference a specific policy or article from your knowledge hub in defined situations so Fin always surfaces the right context in a conversation.
What’s new: Explicit Procedure switching under defined conditions. You can set rules that deterministically trigger a switch to a different Procedure, for example, escalating to a complaints Procedure if specific risk signals are detected mid-conversation.
What’s new: Internal notes for human handoffs. When Fin hands off to a teammate, it can now include internal notes with relevant context so the person picking up the conversation knows exactly what happened and what needs to happen next.
Third, fully agentic behavior. Because real conversations rarely follow the happy path, Procedures let Fin reason through what’s happening and adapt—jumping to the right step or switching Procedures entirely if a customer changes their mind or the issue shifts.
Procedures and Simulations in action: Fin rehearses a food order damage scenario, confirming details and progressing through each trigger. Teams validate complex flows end to end as steps turn green and outcomes are tracked.
What’s new: Automatic Procedure switching. If a customer starts in a billing workflow but then asks about cancelling their subscription, Fin transitions to the relevant Procedure without forcing the customer to restart.
What’s new: Structured data extraction from uploaded files. Fin can now extract structured data directly from PDFs and images uploaded by customers—like invoices, forms, or receipts—and use that data within the conversation. Customers don’t have to copy and paste or repeat themselves.
As MONY Group put it:
“ If a customer starts down one path but their issue turns out to be something else entirely, Fin adapts seamlessly – no more getting stuck in loops or forcing customers into the wrong workflow. ”
Simulations help teams rehearse procedures and verify outcomes before going live. Run all tests or launch a new one to ensure Fin handles tricky customer scenarios—from damage confirmation to refunds and missing subscriptions.
The result is a conversation that feels fluid, but always follows your intended rules.
Making complexity easier to manage is just as important as unlocking new capabilities. Beyond the core updates, we’ve focused on creation, governance, and scale—while keeping ownership with your team.
What’s new: Improved instruction authoring. We’ve made it easier to write, edit, and structure Procedures, so building and updating them takes less time and requires less effort.
What’s new: Reporting on when Procedures trigger, resolve, or hand off. You can now track how Procedures are performing directly within the Procedures UI, seeing exactly when they trigger, when they resolve, and when they hand off to a teammate. This visibility helps you spot issues early and improve over time.
Customer stories from Raylo and Mony Group show how Fin now resolves payment issues and complex claims in-chat, checks account data via APIs, and lifts CSAT to about 94%, highlighting the impact of Procedures and Simulations.
Simulations: Test complex workflows at scale before they reach customers. Simulations let you validate how Procedures will perform before anything goes live, and continuously revalidate as things change. Deploying complex AI can feel uncertain; Simulations remove that uncertainty so you can launch with confidence and iterate safely.
You can simulate full conversations. For any Procedure, choose a user or customer segment and run a complete, multi-turn simulated conversation. You see every step Fin takes, how it applies your rules, reasons through decisions, and where it passes or fails—giving you the observability to debug and fix issues before they ever reach customers.
What’s new: Upload images for richer testing. Simulations now support image uploads, so you can test workflows that involve receipts, invoices, or forms—the same inputs your customers actually send.
What’s new: Clearer visibility into Fin’s reasoning. You can now see exactly how Fin is thinking through each step of a Simulation, making it easier to understand behavior, catch unexpected decisions, and refine Procedures with confidence.
You can also use AI to create, store, and rerun tests. Writing test coverage manually doesn’t scale. Fin’s AI Assistant generates Simulations directly from your Procedures, suggesting realistic edge cases like partial refund disputes, missing invoice uploads, or no subscription found, so you can expand coverage without expanding overhead. All the Simulations you create are stored in a central library. When a product changes, a policy updates, or a Procedure is edited, hit “run all” to instantly check whether anything has regressed. This applies the same rigor to AI automation that engineering teams bring to software testing.
What’s new: AI-suggested Simulations. You can now use AI to generate a full set of Simulations from any Procedure. The AI Assistant suggests realistic variations based on your workflow, so you can build comprehensive test coverage fast.
Customers are already seeing this in production. “Fin can now handle payment-related queries that were never possible before… The impact on CSAT and overall CX has been pretty shocking – the Payment Information procedure CSAT is sitting at ~94%, and CX score is significantly higher than our average.” – Raylo
“Procedures have fundamentally changed what we can achieve with Fin. Previously, complex processes like cashback claim investigations could only be handled through a static form on our website… Now, Fin can handle these sophisticated scenarios in real-time within the conversation itself. It checks account information via API calls, makes complex decisions, and guides customers through the entire claims process dynamically.” – MONY Group
Procedures and Simulations are available now. I’m eager to see how teams use these updates to scale agentic AI, deliver faster resolutions, and raise the bar for customer experience—without sacrificing control, compliance, or quality.
Human-in-the-loop oversight is the fastest and most reliable way I know to elevate AI quality, build user trust, and reduce risk. At HighLevel, my teams treat oversight as a product feature—not an afterthought—because dependable AI experiences come from deliberate design choices across data, models, and people.
When I say “human-in-the-loop,” I mean a system that blends automation with targeted human judgment at key moments: during data curation, prompt engineering, evaluation, deployment, and post-launch learning. This approach turns “AI workflows” into measurable, repeatable processes and keeps me honest about what’s working, what’s drifting, and where a human safety net must step in.
Architecturally, I start with a retrieval-first pipeline to ground outputs in trusted knowledge, then wrap it in guardrails. Deterministic preprocessing, careful prompt engineering, and post-processing validators catch obvious failure modes. Confidence thresholds and policy checks route ambiguous or sensitive cases to a human reviewer, while clear, auditable traces show why the system chose automation versus escalation. This balance supports reliability at scale while preserving agility for “agentic AI” patterns when they add value.
Quality is only real if I can measure it, so I build with eval-driven development from day one. I maintain golden datasets, rubric-based scoring guidelines, and an automated evaluation harness that runs on every change to prompts, models, or data. Pre-production gates protect against regressions, while production telemetry surfaces drift by segment and use case. When it’s time to run experiments, I use A/B tests sized with a minimum detectable effect (MDE) to avoid overfitting to noise.
Operationally, I optimize for outcomes, not output. I track task success rate, time-to-resolution, safety violation rate, hallucination rate, and cost-to-serve, then connect these to outcomes vs output OKRs. The signal I want is simple: are we reliably solving the user’s job-to-be-done with lower effort and higher confidence? If not, I tighten prompts, refine retrieval, or expand human review where it pays off most.
Risk governance is non-negotiable. I design with privacy-by-design and data governance from the start—role-based access, audit trails, PII redaction, and red-team tests for safety. Clear reviewer playbooks and calibration sessions reduce bias and ensure consistent decisions. These practices aren’t bureaucracy; they’re how I operationalize AI risk management while maintaining velocity.
Teams make or break this model. I empower product trios to own the full lifecycle—discovery, build, and learning—so feedback loops close quickly. In-product feedback widgets, reviewer queues, and incident management playbooks help us respond in hours, not weeks. Over time, human review becomes a targeted scalpel rather than a blanket requirement as the system learns and improves.
Economics guide the level of oversight. I treat each workflow like a portfolio: where the value of accuracy is high and ambiguity is common, I route more to humans; where tasks are simple, frequent, and well-bounded, I automate aggressively. The goal isn’t zero humans—it’s optimal humans, deployed precisely where their judgment compounds ROI.
If you’re getting started, begin with one high-impact workflow, establish your golden set and evaluation rubric, and wire in a simple review queue. Prove the lift, then scale. In the short video above, I walk through the patterns I use to design these loops, measure quality with rigor, and ship AI that teams—and customers—can trust.
For years, I chased the elusive goal of delivering a perfect customer experience. Today, with AI embedded in our support operations, that standard is finally within reach—and it’s reshaping how we prioritize, design, and scale service.
In “The 2026 Customer Service Transformation Report,” teams report early, tangible wins from AI: faster responses, higher efficiency, and consistent coverage across languages and time zones. Those gains create the capacity we’ve always needed. The more we push the technology, the more quality improvements we unlock.
This marks a fundamental shift. As AI takes on more, our focus can finally move from firefighting to crafting the customer experience. When the AI is working, the measure of success becomes how well it’s working—across accuracy, tone, resolution, and end-to-end journey quality.
I’ve seen this transformation firsthand. Mature AI deployment gives my team “breathing room,” so we can design for consistently excellent outcomes rather than obsess over deflection. That means widening access to support, removing friction on the path to resolution, and anticipating customer needs before they escalate.
In our own support organization, we opened support to trial customers, accelerated first response times, and added consultative sessions during onboarding. We absorbed a 300% increase in total demand without adding headcount—made possible by deep integration of an AI Agent and a disciplined AI strategy.
Teams with mature customer service deployments are nearly three times likelier to say they always meet increasing expectations—27% vs 9% at initial rollout—highlighted by bold orange and gray comparison bubbles.
Across the industry, the pattern is similar. When teams initially deploy AI, only 9% say they can always meet customer expectations. That number triples as teams reach a mature level of deployment. Even as expectations rise, the organizations that deeply integrate AI—complete with clear ownership, robust instrumentation, and continuous improvement loops—are the ones most likely to meet (and exceed) the bar.
Looking ahead to 2026, I expect omnichannel consistency to become a key differentiator. The data shows planned investment is distributed nearly equally across chat, email, and social messaging (36% each), closely followed by phone/voice (31%). The question is no longer “Which channel should we optimize?” but “How do we deliver a consistent, AI-powered experience everywhere our customers are?”
Teams that solve for omnichannel consistency will bridge the long-standing gap between what customers expect and what support can deliver. Every touchpoint becomes an opportunity to exceed expectations and build durable trust.
Consider Clay, a team that scaled support without sacrificing quality. Support is one of their main growth drivers, and as their customer base expanded, ticket volume surged. Early on, they concentrated much of their effort in Slack, cultivating close, transparent community relationships. But relying on a single channel created friction as they grew; customers wanted the flexibility of email and in-app chat, and Clay needed to deliver the same high standard everywhere.
Where AI investment is headed for customer service in 2026: chat, social, and email lead at 36%, with phone/voice close behind at 31%. A bold visual snapshot of shifting channel priorities in CX.
By unifying their support experience with an AI Agent, Clay brought consistency across channels. Today, AI is involved in 90% of all queries and handles half of Clay’s total volume, upwards of 7,000 queries a month. First response rates improved significantly, freeing the team to focus on proactive, high-impact work.
That work includes identifying content gaps for education and content marketing, reaching customers before they need to ask for help, and surfacing feature requests and recurring challenges to product teams. Clay proves that when support is truly great, it becomes a competitive edge.
So how do you build a superior customer experience with an AI Agent? Here are five principles I use when scaling toward mature deployment.
1) Treat customer experience like a product. Treating support as a product means designing, building, and managing the support experience with the same rigor as your core product. You define goals (faster onboarding, higher CSAT or CX Score, lower churn). You map flows (AI starts the conversation, human handovers, proactive nudges). You instrument the journey (track handoffs, drop-offs, success states). You run tests and ship improvements (tone tweaks, fallback paths, training updates). You own the outcomes (gather feedback, measure performance, use insights to continuously improve the system).
Leaders are racing ahead with real AI in support. Explore the 2026 Customer Service Transformation Report to see where deployment is stalling, benchmark your team, and get practical steps to scale automation that delights.
2) Lead with AI, back with humans. AI isn’t replacing the human touch. It’s redefining when, where, and how it’s most valuable. In a scaled model, AI is the first responder and the end point for most conversations. Humans step in where they add the most value—particularly during high-stakes issues—and those handoffs should feel seamless. Meanwhile, your team focuses on improving AI performance and optimizing the end-to-end journey.
3) Be proactive. Use AI to anticipate needs, guide customers before problems arise, and nudge them toward successful outcomes. This is where customer support AI strategy shines—moving from reactive triage to journey orchestration that protects momentum and builds trust.
4) Build for trust. Many customers still carry the legacy of clunky chatbots that delivered vague answers and dead ends. You earn trust by showing that your system works. Don’t hide your AI Agent behind layers of “choose an option.” Get customers to the AI quickly, demonstrate real problem-solving, and ensure that when a human is needed, they join with full context to resolve complex issues efficiently.
5) Make it feel personal. Your AI Agent represents your brand. The way it speaks, follows policies, and responds matters. Use tone control, fallback logic, and language preferences to align the experience to your standards. Consistency builds trust; personality builds connection and loyalty.
Perfect really is possible. With deep AI implementation, you can scale comprehensive, fast, and personal support across channels—so customers feel supported not just when they reach out, but throughout their journey. That’s the promise of modern AI workflows in support, and it’s what will separate leaders from laggards in the years ahead.
Your team may already use AI to draft emails, summarize calls, research accounts, and answer website questions. Yet the lead still waits in a queue, the seller still reconstructs context, and the customer still repeats the same information after every handoff.
If you are deciding how to make go-to-market genuinely AI-native, do not start with another list of tools. Decide which part of the revenue journey can run as a complete, observable workflow: AI handles defined decisions and actions, humans take over at explicit boundaries, and both work from the same customer state.
Redraw the revenue workflow before automating its tasks
Adding an assistant to every department can make individual tasks faster without making the revenue system faster. Marketing produces more content, sales receives more research, and customer success gets more summaries, but the queues and handoffs between those functions remain intact. More output can even make those bottlenecks worse.
An AI-native workflow has a different unit of design. It owns a bounded outcome over time. It can observe an event, retrieve approved context, choose among permitted actions, update the CRM, evaluate what happened, and either continue or escalate. The distinction matters: generating a follow-up email is a task; noticing that a qualified buyer has gone quiet, selecting the appropriate follow-up, sending it under policy, recording the attempt, and changing course based on the response is a workflow.
Map one live customer journey before discussing models or vendors. For every transition, write down six things:
Trigger: What observable event starts the work? Examples include a demo request, an unanswered question, a completed trial action, or a missed follow-up.
State: What must be known before anyone acts? Include the account, buyer, stage, previous interactions, consent, product usage, and open commitments that matter.
Decision: What choice is being made? Qualification, routing, next-best action, escalation, or disqualification should not be hidden inside a vague prompt.
Action: What is the agent actually allowed to do? Drafting, sending, booking, calling, demonstrating, updating a field, or creating a human task are different permission levels.
Evidence: What will prove that the action was appropriate and completed? Preserve retrieved passages, tool results, timestamps, policy checks, and the resulting CRM change.
Exception owner: Who takes responsibility when confidence is low, the buyer objects, the data conflicts, or the request falls outside policy?
This map exposes where AI can remove elapsed time rather than merely reduce typing. Baseline the current path using measures you already trust: time between stages, abandonment points, manual touches, repeated discovery, and incomplete CRM records. Then choose one customer outcome, such as completing a qualified next step, instead of treating generated messages as success.
My test is simple: if the proposed system cannot identify its current state, show why it acted, and recover from a failed action, it is still a feature. It is not yet part of the revenue operating system.
Start with a bounded inbound motion, then earn more autonomy
I would usually start where the buyer has already expressed intent. An inbound visitor asking a product question or requesting a demonstration gives you a clear trigger, an identifiable job, and a natural human fallback. You can observe whether the interaction advances the buyer without asking an agent to manufacture demand across an ambiguous market.
A strong first workflow has five properties:
The entry event and desired next state are unambiguous.
The agent can answer from an approved and maintainable body of knowledge.
Most actions are reversible, or a human can approve them before execution.
Failure and frustration can be detected quickly.
The business outcome appears in a system of record rather than a separate AI dashboard.
Inbound qualification, guided product education, demo support, meeting preparation, and structured follow-up often fit these conditions. A practical early implementation can be deliberately modest: a voice interaction, reusable product demonstrations, and a retrieval-first knowledge layer. Retrieval gives the agent current, company-approved material without forcing every sales fact into a prompt, and it gives evaluators evidence against which to judge an answer.
Treat the interaction surface as part of the workflow, not as decoration. In one documented implementation, adding a realistic avatar changed how prospects behaved: they interrupted, probed, and requested demonstrations in ways associated with a live sales conversation. That is evidence that an interface changes the behavior it invites, not proof that every buyer or sales motion needs an avatar. Test chat, voice, and video against the buyer’s actual job. Do not choose the most human-looking interface by default.
Expand autonomy in gates rather than with one large launch:
Shadow: The agent recommends an answer, decision, or next action while a human remains responsible for execution. Use disagreements to build the first evaluation set.
Constrained execution: The agent handles approved questions and actions, writes every result to the CRM, and routes exceptions to a named person.
Bounded workflow ownership: The agent can continue across interactions and days, but only inside an explicit state machine, policy envelope, and escalation contract.
Adjacent expansion: Reuse proven capabilities in another stage, such as onboarding or customer success, only after the first workflow is stable and measurable.
Gate movement on evidence, not enthusiasm or a calendar date. The agent should not receive a new action merely because it can generate plausible language. It should receive that action when you can detect a bad decision, contain its consequence, and restore the customer journey.
Give agents distinct jobs and humans a real handoff contract
A single prompt that qualifies, pitches, retrieves facts, controls tools, remembers history, evaluates itself, and decides when to escalate becomes difficult to test. It also mixes goals that can conflict. A persuasive sales response and a conservative policy check should not compete for attention in the same undifferentiated instruction block.
A useful architecture separates five responsibilities:
Knowledge or creator agent: Turns approved documentation, training material, and transcript patterns into versioned playbooks and retrievable knowledge.
Conversation agent: Handles the live interaction, asks the next appropriate question, and stays within the current objective.
Workflow orchestrator: Maintains state across channels and time, selects the next permitted step, invokes tools, and pauses when an exception occurs.
Evaluator: Scores the interaction for grounding, policy compliance, conversation quality, sentiment, and task completion.
Human owner: Resolves ambiguity, negotiates unusual terms, restores trust, and changes the playbook when the system exposes a recurring gap.
Specialized conversation roles can also help manage latency, context limits, and model weaknesses. Greeting, discovery, qualification, and pitching do not necessarily need identical instructions or context. The important design move is not the number of agents; it is the ability to isolate a responsibility and test it. Deterministic paths can handle predictable stages while orchestration manages contextual departures.
Do not split a workflow into a swarm merely because multi-agent architecture sounds advanced. Start with the fewest independently testable components. Split one when its context becomes noisy, its latency threatens the experience, its permissions need separate control, or its failures require a different evaluation method.
The orchestrator should persist a compact state record that another worker can understand. At minimum, capture the current objective, stage, known facts, supporting evidence, confidence, unresolved questions, commitments already made, next permitted action, and current owner. Keep the transcript available, but do not force every downstream decision-maker to reconstruct state from raw conversation history.
A human handoff is a product contract, not an emergency notification. Define its triggers before launch. Useful triggers include:
The agent cannot find approved evidence for a material answer.
Confidence falls below the level you have defined for that action.
The buyer repeats a correction, expresses frustration, or explicitly requests a person.
The request involves commercial, security, legal, or contractual judgment outside the approved playbook.
A tool fails, CRM state conflicts with the conversation, or the proposed action would duplicate an existing commitment.
The receiving person needs more than a transcript link. Send the buyer’s goal, the current stage, facts already established, the reason for escalation, supporting evidence, actions attempted, promises made, urgency, and the decision the human must make. Pause further automated outreach until ownership is acknowledged. Otherwise, the agent can send a cheerful follow-up while a seller is handling a sensitive objection.
After the human resolves the case, specify whether control returns to the workflow and what state must change first. That return path is where many pilots quietly become permanent manual operations.
The CRM must carry this shared state in both directions. Agents should read the latest account context and write back decisions, actions, outcomes, and evidence. A system that conducts a convincing conversation but leaves the record incomplete creates invisible work for the next person. Tight CRM integration and persistent workflow orchestration are what turn an interface into an operating capability.
Use evaluations and revenue governance as the control plane
A polished demonstration proves that a workflow can succeed once. Revenue leaders need evidence that it succeeds repeatedly, fails visibly, and improves without silently regressing. That requires an evaluation system tied to release decisions.
Build the control loop in this order:
Define the failure taxonomy. Separate unsupported facts, policy violations, missed discovery, poor routing, broken tools, excessive latency, incorrect CRM updates, weak handoffs, and incomplete outcomes. A single quality score hides the repair you need.
Create a representative evaluation set. Include common interactions, important segments, known edge cases, adversarial requests, tool failures, and examples that should trigger escalation. Label the expected action and unacceptable actions, not only an ideal sentence.
Review production conversations aggressively at the start. One practical deployment pattern reviews every interaction during early operation and tapers toward a sample of about 5% as confidence grows. The reduction is earned through observed quality; it is not a default schedule. Customer review, evaluator scoring, and sampling can operate as one quality loop.
Turn failures into regression tests. When a human corrects an answer, routing decision, or handoff, add a durable test before changing the prompt or playbook. Otherwise, fixing one conversation can break another without detection.
Release progressively. Use proof-of-concept validation, controlled exposure, A/B rollout where appropriate, CRM logs, and dashboards. Preserve a rollback path for prompts, models, playbooks, tools, and policies.
Expand authority only after two kinds of evidence agree. Agent quality must remain acceptable, and the intended business outcome must improve without shifting damage into complaints, bad-fit pipeline, downstream rework, or customer churn.
Your dashboard should distinguish system quality from business performance. Both are necessary, and neither can substitute for the other.
Qualified progression, completed bookings, stage movement, activation, retention, and downstream rejection of poor-fit opportunities
Operating health
Can the capability run and improve reliably?
CRM completeness, failed actions, recovery, human review load, overrides, version history, and cost per completed outcome
Do not reward the agent for conversion alone. A system can raise a local conversion metric by overpromising, qualifying weak opportunities, or making escalation harder. Review performance by segment and release version, and keep quality and downstream outcome guardrails beside the target metric.
Governance also needs named owners. The CRO should own the end-to-end revenue outcome and the interlocks across marketing, sales, solutions engineering, onboarding, and customer success. Product and AI leaders should own agent behavior, experience, evaluation infrastructure, and release gates. Revenue operations should own CRM state, definitions, attribution, and operational dashboards. Functional leaders should own their playbooks and exception policies. Humans in the workflow should own judgment where trust, negotiation, and ambiguity matter.
Centralize the parts that must remain consistent: data definitions, core tooling, pricing guardrails, evaluation standards, and foundational enablement. Let segment plays, partner motions, and contextual field execution stay closer to the teams that understand them. This avoids two common extremes: every function buying its own disconnected agent, or a central AI group becoming the queue for every revenue experiment.
Pair gated releases with a 24-26 month design horizon. The longer view is not a promise that you can forecast models or markets precisely. It forces you to ask what breaks at higher volume, which capabilities must remain centralized, how roles will change, and what data or evaluation debt would block the next level of autonomy. The release in front of you can remain small while the operating architecture anticipates scale.
The first leadership review should produce five concrete artifacts: a workflow map, an agent charter, a human handoff contract, an evaluation set, and a dashboard with a named decision-maker for every release gate. If the meeting ends with only a vendor shortlist, the transformation has not yet been designed.
Key takeaways
If an agent cannot read and update shared customer state, it is an interface attached to the revenue system, not a worker inside it.
If the handoff criteria and return path are unclear, the agent’s autonomy is already too broad.
If production failures do not enlarge the evaluation suite, the organization is collecting incidents rather than compounding learning.
If a buyer must repeat discovery after escalation, the context architecture has failed even when the AI behaved politely.
If the roadmap is organized around tools instead of revenue-state transitions, no executive truly owns the transformation.
At your next go-to-market review, choose one live inbound path and walk a real lead through it from trigger to next customer outcome. Assign state, evidence, permissions, and an exception owner at every transition. Automate only the steps whose failure you can detect and recover from. That is how you earn the right to give AI more of the revenue journey.
Over the past few years, I’ve led cross-functional teams to deploy agentic AI in production, and I’ve learned that success rarely hinges on the model alone. It comes from methodically designing the right workflows, instrumenting every step, and building a feedback loop that compounds. Learn how companies like Replit are consolidating workflows, creating one-person departments, and building systems for scale with Amplitude.
When I talk about AI agents, I’m describing software that behaves like a focused teammate—owning a clear job to be done end-to-end. In practice, that means consolidating fragmented tasks into a single accountable “one-person department,” then giving it the context, tools, and analytics to perform reliably. This is how agentic AI moves beyond demos into durable business impact.
I start with outcomes, not algorithms. I map a driver tree from business goals (e.g., lower response time, higher activation, better retention) to the specific moments an agent can influence. This outcome-first alignment keeps scope tight, informs guardrails, and grounds the value proposition in measurable change instead of vanity metrics.
Next, I define the workflow the agent will fully own. I look for high-volume, rules-adjacent processes—think lead qualification, support triage, or billing inquiries—where clear decision criteria already exist but human time is the bottleneck. I document triggers, inputs, decision points, and handoffs, then design the ideal-state flow the agent will run autonomously, with transparent escalation paths to humans.
On architecture, I favor a retrieval-first pipeline to keep responses accurate and current. I scope the knowledge base, implement context window management, and standardize tools the agent can call (search, CRM actions, ticket updates). For teams new to this, I coach “LLMs for product managers” fundamentals so we make sensible trade-offs between speed and reliability rather than chasing model-of-the-week headlines.
Instrumentation is where the system becomes self-improving. I use Amplitude analytics and an Agent Analytics schema to track intent detection, tool usage, resolution rate, time-to-resolution, deflection, and escalation causes. A unified analytics platform lets me connect agent outcomes to core product metrics—activation, retention, and conversion—so we can see the real revenue and experience impact, not just local efficiency gains.
To validate impact, I run A/B testing when traffic allows, setting a minimum detectable effect (MDE) upfront to avoid inconclusive reads. In lower-volume scenarios, I lean on eval-driven development: curated test sets for edge cases, scenario-based regression suites, and error taxonomies that accelerate iteration. Feature flags let us stage capabilities safely (shadow mode, assistive, autonomous) while we monitor deltas before full rollout.
Reliability and trust are designed in from the start. I apply AI risk management practices—privacy-by-design, data governance, and policy-aligned prompt templates—paired with observability to trace decisions. Clear escalation policies, incident management runbooks, and human-in-the-loop checkpoints ensure the agent fails safe, not silently.
Shipping cadence matters. I use CI/CD to increase deployment frequency, keep prompts and tools versioned, and gate risky changes with targeted rollouts. As patterns stabilize, we scale horizontally to new use cases, sharing core capabilities (retrieval, analytics, guardrails) as a platform. This is how “one-person departments” multiply without multiplying overhead.
Change management closes the loop. I partner with product trios and frontline teams to co-design prompts, set acceptance criteria, and define what “good” looks like in plain language. In-app guides and product tours introduce the agent’s role and limits, and structured feedback channels feed directly into our discovery and iteration rhythm.
The throughline of this playbook is simple: treat agents like real teammates with a job description, operating procedures, and performance reviews. With disciplined workflow design, a retrieval-first pipeline, and outcome-level instrumentation in Amplitude, agentic AI stops being a science project and starts compounding into durable product-led growth.
Inspired by this post on Amplitude – Perspectives.
You have customer interviews, an AI prototype, and a launch request. What you may not have is a defensible chain connecting them. The prototype can look convincing while the team still disagrees about the customer problem, the acceptable failure rate, the limits of automation, and what should happen when the model or a connected tool fails.
A durable AI product workflow makes those decisions explicit. It connects customer evidence to a bounded opportunity, the opportunity to an interaction model, that model to an evaluation contract, and the contract to a guarded production release. You should be able to trace every automated action backward to a customer need and forward to a metric, an owner, and a recovery path.
Turn interviews into an opportunity map, not a feature request
AI products often go wrong before anyone writes a prompt. A customer describes a slow or frustrating task, someone proposes an assistant, and the proposed interface quietly becomes the problem definition. The team then tests whether it can build the assistant instead of whether solving that part of the workflow changes the customer’s outcome.
Start by defining the discovery boundary. Name the user, the workflow, the outcome the user is trying to reach, and the part of that outcome your product could reasonably influence. Keep interviews in the same outcome or product space when you synthesize them. A small batch of three interviews can be enough to produce a useful first draft, but it is not a universal saturation threshold or proof that you understand the market.
The sequence of synthesis matters. Analyze each interview on its own before looking for patterns across interviews. That preserves the situation, sequence, and meaning around each customer’s comments. If you combine transcripts immediately, repeated vocabulary can appear more important than the underlying context, and an unusual but consequential problem can disappear into the average.
Write the outcome anchor. State whose behavior or result should change. Avoid a feature-shaped outcome such as “increase use of the AI assistant.” A better outcome describes progress in the customer’s work.
Create one snapshot per interview. Capture the customer’s goal, the relevant sequence of events, key moments, obstacles, current workaround, and evidence supporting each inferred opportunity.
Separate observation from interpretation. Preserve what happened and what the customer said separately from the team’s explanation of why it happened. Label uncertainty instead of filling gaps with generated prose.
Synthesize across snapshots. Look for shared opportunities, meaningful differences, dependencies, and contradictions. Similar wording does not automatically mean the same need.
Organize opportunities before proposing solutions. Build an opportunity solution tree or equivalent map that connects the product outcome to customer opportunities. Keep solution ideas outside the opportunity labels.
Review the generated structure as a team. Ask what was merged incorrectly, what was missed, what lacks evidence, and which branch reflects a solution disguised as a need.
AI is useful here as a first-pass analyst, not as an authority. It can extract moments, propose opportunity statements, and suggest a hierarchy. Human reviewers contribute product context, recognize important exceptions, and challenge confident-looking inferences. The strongest practical model is an AI-generated draft that the team refines.
Your exit gate for discovery is not a polished tree. It is agreement on a selected opportunity, the evidence behind it, the customer outcome it should influence, and the opportunities deliberately excluded from the current scope. If the team cannot explain those choices without mentioning a model or interface, it is not ready to prototype.
Choose assistance or autonomy before choosing the architecture
The next decision is not which model to use. It is what responsibility the product will accept. An LLM can generate or classify content. An agent wraps model behavior in a workflow that plans, uses tools, retains relevant state, and attempts to complete an outcome. That difference changes the customer promise, the evaluation plan, the permission model, and the consequences of failure.
Decision
Copilot
Agent
Best task shape
High-context work that benefits from judgment, nuance, or brand voice
Bounded, tool-heavy work with a verifiable completion state
Customer promise
Drafts, explains, recommends, or accelerates
Completes an agreed task within a defined scope
Human role
Reviews and commits the result
Sets policy, handles exceptions, and approves sensitive actions
Default permissions
Read, retrieve, and propose
Narrowly scoped tool access, including only the writes required for the task
Primary proof
Useful, grounded output that improves the user’s work
End-to-end task success without unacceptable actions or loops
Failure consequence
A poor suggestion reaches the reviewer
A poor decision can propagate into another system
When the task still depends on tacit knowledge or subjective review, start with a copilot. When it is bounded, tool-heavy, and objectively checkable, consider an agent. The safer product progression is to start assistive and grant autonomy only after success is measurable. Autonomy should be earned capability by capability, not declared at the product level.
You can make that progression concrete without redesigning the entire experience. Let the product draft first. Then let it recommend a plan and show the evidence behind the recommendation. Next, allow reversible actions through a narrow tool whitelist. Keep approval immediately before actions that affect customers, money, permissions, or durable data. Expand the scope only when production evidence supports the previous boundary.
Once the responsibility is clear, define the architecture around it:
Authoritative context: retrieve relevant product, account, policy, or workflow information before asking the model to decide. A retrieval-first pipeline reduces dependence on whatever happens to be encoded in model weights.
Explicit scope: state the role, allowed objectives, prohibited actions, and conditions that require escalation.
Controlled tools: expose only the operations needed for the selected job. Apply unit limits and validate tool inputs outside the model.
Deliberate memory: separate temporary working state, durable customer facts, and governing policy. Do not treat the entire conversation history as an undifferentiated memory store.
Visible checkpoints: show the user what will happen, what data will be used, and which action requires approval.
Traceable execution: record retrieval results, model and prompt versions, tool calls, approvals, guardrail events, and final task status.
This architecture is more durable than a large prompt because each component has a distinct failure mode and owner. Retrieval can be evaluated for evidence quality. Tools can be tested deterministically. Policy can be reviewed independently. The model remains important, but it no longer carries responsibilities that ordinary software can enforce more reliably.
The exit gate is a written responsibility boundary. The team should be able to say what the product may read, what it may write, what it must never do, when a person intervenes, and how successful completion is verified. If any answer is “the model will decide,” the boundary is still incomplete.
Write the evaluation contract before optimizing the prompt
A compelling demo proves that a path can work. It does not establish how often it works, which inputs break it, whether its evidence is trustworthy, or whether it completes the customer’s job at an acceptable cost. Prompt iteration without an evaluation contract tends to optimize whatever the last reviewer noticed.
Write the contract in product language. For each target task, define the eligible input, the expected outcome, the evidence the product may use, allowed actions, prohibited outcomes, completion criteria, escalation conditions, and fallback. Add latency and cost limits chosen for your product economics. There is no universal threshold that makes an AI workflow production-ready; the important discipline is setting the threshold before seeing launch results.
Build the evaluation set from discovery evidence. Include representative customer inputs, important workflow variations, ambiguous cases, missing context, conflicting instructions, tool failures, and requests the product must refuse or escalate. Remove or protect sensitive data according to your governance rules. Every case should identify the acceptable outcome, not merely an ideal sentence, because multiple responses may solve the same job.
For copilots, measure the quality of assistance
Time to first token: how long the user waits before the response begins.
Response latency: how long the useful result takes to complete.
Groundedness: whether material claims are supported by the authoritative context supplied to the model.
User satisfaction: whether the assistance was useful in the actual workflow, not merely fluent.
Task impact: whether the user completes the selected job faster, with less effort, or with fewer corrections, using the outcome defined during discovery.
For agents, measure the whole execution
Task success rate: successfully completed eligible tasks divided by all eligible attempts. Define completion in the customer’s system of record where possible.
Steps per task: the number of model and tool steps required to finish. A rising count can expose inefficient planning or repeated work.
Tool error rate: failed, rejected, or malformed tool calls relative to attempted calls.
Loop detection: executions stopped because the agent repeated actions or failed to make progress.
Guardrail triggers: attempts blocked or redirected by policy. A trigger is diagnostic evidence, not automatically a success or a failure.
Human escalation: tasks handed to a person because the agent lacked permission, confidence, context, or a valid recovery path.
Cost per successful task: total execution cost divided by successful completions. Cost per request can hide expensive retries and failed runs.
Containment rate: eligible tasks completed within the automated workflow without human handling. Publish the eligibility and escalation rules with the metric so teams do not improve containment by narrowing the denominator invisibly.
These agent analytics complement rather than replace end-to-end task success. A fast response can still be wrong. A low tool error rate can coexist with a bad plan. High containment can be harmful if the agent completes the wrong task. Choose one outcome metric, pair it with quality and safety constraints, and retain the diagnostic metrics needed to find the cause of failure.
Route failures to the component that can fix them. Unsupported claims point first to retrieval and grounding. Correct plans with failed actions point to tool integration. Repeated steps point to orchestration or stopping logic. Frequent, legitimate escalations may mean the autonomy boundary is too broad. High model scores with low customer satisfaction should send the team back to the opportunity definition or user experience.
The exit gate is a versioned evaluation suite with release criteria, prohibited outcomes, an approved cost ceiling, and named escalation rules. Run it against every material change to the model, prompt, retrieval configuration, tool contract, or policy. Treat prompts and evaluation cases as product assets under version control, not as text pasted into a dashboard.
Release through gates and design the failure path
Deployment is where an AI capability becomes a product promise. The team now has to manage model variability, external tool behavior, changing knowledge, permissions, cost, and customer expectations at the same time. A launch plan that covers only the happy path is unfinished.
Put the capability behind a feature flag. Separate deployment from exposure so the team can stop new executions without waiting for a code release.
Open a gated beta around one bounded job. Limit the eligible users, tool permissions, data scope, and advertised promise. Make it clear whether the product recommends an action or performs it.
Use a canary for broader production traffic. Expand exposure gradually while comparing task success, guardrail events, tool errors, latency, escalation, and cost per successful task with the release criteria.
Change one material layer at a time when practical. Simultaneous changes to the model, prompt, retrieval index, tools, and policy make regressions difficult to attribute.
Expand only after the previous boundary is stable. More users, more tools, and more autonomy are separate risk decisions. Do not bundle them into one rollout.
Keep rollback and fallback distinct. Rollback restores a known model, prompt, policy, or tool version. Fallback gives the customer a safe alternative when the AI path is unavailable.
Model unavailable: preserve the user’s work, explain that automation is unavailable, and offer the established manual path.
Retrieval unavailable or evidence missing: do not silently generate an ungrounded answer. Ask for the missing context, provide a limited response, or escalate.
Write tool fails: stop, report the actual system state, and reconcile before retrying. Blind retries can duplicate durable actions.
Execution stops making progress: terminate the loop at the configured limit and hand over the trace rather than consuming resources indefinitely.
Policy or permission check fails: block the action, preserve the audit record, and route the user to an authorized path.
Tool behavior changes: disable the affected capability until its contract and evaluation cases pass again.
Privacy and auditability belong in the release gate, not in a later compliance review. Document what customer data enters prompts, retrieval, memory, and logs; who can access it; how long each class is retained; and how deletion propagates. For actions affecting customers, money, permissions, or durable data, preserve enough detail to reconstruct the input, retrieved evidence, model and prompt version, tool parameters, approval, guardrail result, and final system state.
The operating stack also needs an ownership decision. Build the workflow logic, data model, and user experience that encode your differentiated value. Consider buying undifferentiated capabilities such as observability, prompt versioning, red-team infrastructure, and policy enforcement when an external component meets your control and governance needs. This build-versus-buy boundary keeps product attention on the parts customers actually choose you for without treating commodity infrastructure as strategically unique.
The production exit gate should require a visible scope statement, passing evaluations, a feature flag, a rollback target, a customer-safe fallback, usable audit traces, an incident owner, and a tested escalation route. If the team cannot explain what the customer sees during failure, it has not finished designing the feature.
Keep discovery, evaluation, and production in one learning loop
Once the product is live, production behavior becomes new discovery input. That does not mean replacing customer conversations with dashboards. Metrics show where the workflow breaks; customer evidence explains what the break means and whether fixing it matters.
Review failures against the original opportunity map. Concentrated escalation around one scenario may reveal an opportunity that was hidden during initial synthesis. High groundedness with low satisfaction may indicate that the product answered accurately but tackled the wrong job. A growing step count may expose orchestration waste, while a rising tool error rate points to integration reliability. If cost per successful task increases, inspect failure and retry paths before making the model cheaper; optimizing unit cost cannot rescue an unsuccessful workflow.
Every meaningful production failure should produce at least one durable change: a corrected opportunity assumption, a new evaluation case, a narrower permission, a tool-contract test, a policy update, a clearer interaction, or a revised fallback. That is how customer discovery and operational learning remain connected instead of becoming separate product and engineering rituals.
Key takeaways
Synthesize each customer interview separately before looking across interviews, then review the AI-generated opportunity structure with human judgment.
Select a customer opportunity before selecting the AI interface. A fluent prototype is not evidence that the underlying job matters.
Use a copilot for judgment-heavy work and consider an agent only for bounded, tool-heavy tasks with verifiable completion.
Define task success, prohibited outcomes, escalation, cost, and fallback before optimizing prompts or choosing a model.
Measure copilots as assistance and agents as end-to-end execution. Do not mistake latency, containment, or tool-call success for customer success.
Release behind flags, expand through gated exposure, and rehearse rollback, fallback, and incident paths before granting more autonomy.
At your next AI product review, ask to see the outcome and opportunity map, the responsibility boundary, the evaluation contract, and the rollout and recovery plan. If one is missing, pause the launch decision at that handoff. Closing that gap is usually more valuable than adding another prompt, tool, or autonomous step.
In my day-to-day building AI products, I’ve learned a simple truth: a single model can be brilliant, but a coordinated team of specialized agents is what consistently ships outcomes customers trust. That’s the promise of multi-agent systems—multiple AIs with distinct roles collaborating inside robust AI workflows to deliver accuracy, speed, and resilience you can’t get from a lone model.
Think of a multi-agent system as a well-run product trio for machines: a planner decomposes the job, specialists execute focused tasks, a reviewer checks quality, and an orchestrator keeps everyone aligned. This agentic AI approach mirrors how high-performing teams work—divide complex problems, play to strengths, and create tight feedback loops.
When does one AI stop being enough? Whenever tasks require tool use, domain retrieval, multi-step reasoning, or policy adherence under real-world constraints. In those moments, specialized agents shine—one for search using a retrieval-first pipeline, another for reasoning, another for action execution, and a final one for validation. The result is better accuracy with manageable latency and cost.
The core architecture I rely on starts with a planner that breaks a goal into steps, followed by execution agents equipped with tools and grounded context. I pair this with context window management to keep prompts lean and relevant, and I insert a verifier (or critic) to catch logic slips and policy violations before results reach customers. A lightweight orchestrator coordinates handoffs and retries to keep the whole flow resilient.
To make this production-grade, I treat observability as non-negotiable. Agent Analytics helps me see which agents are adding value versus adding latency, where failures cluster, and how prompts drift over time. From there, eval-driven development gives me measurable confidence: I codify representative tasks, run offline and shadow evaluations, and only promote changes that move accuracy and safety in the right direction.
Governance is equally critical. I design privacy-by-design from the start, restrict data movement with strong data governance, and enforce policy constraints inside the workflow rather than after the fact. This includes red-teaming failure modes, rate-limiting tools, and capturing immutable traces for audits and post-incident reviews—habits borrowed from SRE culture that map well to AI systems.
On the practical side, prompt engineering remains foundational, but it’s the system design that converts clever prompts into reliable outcomes. Tool access, retrieval quality, memory strategy, and error handling matter more than wordsmithing alone. I’ve found that small prompt improvements are amplified when the surrounding workflow is sound—and are overwhelmed when it isn’t.
If you’re just starting, begin with a narrow use case and a minimal set of agents—planner, executor, and verifier—then expand. Use continuous discovery with real users to learn where the workflow fails in the wild, and iterate with tight release cycles. Treat every agent like a microservice with clear contracts, test coverage, and metrics, and you’ll unlock compounding gains without losing control.
The payoff is tangible: faster shipping cycles, fewer regressions, and outcomes customers can actually rely on. When stakes are high and ambiguity is real, one AI is often a talented soloist—but a disciplined ensemble of agents is how I deliver dependable, scalable value at product velocity.
Over the last year, I’ve had the same conversation with a lot of support leaders.
They’ve deployed AI and are seeing initial efficiency gains, but want to push beyond these early results and achieve meaningful transformation.
When AI is first introduced, the gains show up quickly. Teams resolve higher volumes of queries, free up capacity, and deliver faster responses. But the real opportunity for impact extends well beyond those initial wins. As AI becomes more deeply integrated into support operations, taking on harder, more complex work, those results compound, new ways to create and measure value open up, and the economics of support change entirely. That shift is where I spend most of my time with leaders—turning early efficiency into durable business value.
This sits at the heart of “The 2026 Customer Service Transformation Report.” In this reflection, I explore how deeper integration compounds impact and why that makes business value easier to articulate across the organization—especially to finance and product peers who need to see outcomes, not just output.
The teams going deeper are seeing higher returns. The research shows that 62% of support teams have seen their customer service metrics improve since implementing AI, with early wins showing up most clearly in speed and efficiency. But for teams that have reached mature deployment (where AI is fully integrated into operations) that number jumps to 87%.
As AI programs advance, measurement confidence surges. This chart shows how ROI tracking rises from 35% in exploring to 70% in mature deployments—evidence of a widening execution gap in customer service.
The same pattern holds for the ability to measure ROI. Among teams in early exploration, just 35% say they can measure their return on AI investment, but for teams at the mature deployment stage, that rises to 70%. In my experience, this is the moment the conversation shifts from “is AI working?” to “how much leverage are we creating?”
As AI becomes more embedded in support workflows, what teams choose to measure starts to change. In the early stages of deployment, ROI is typically understood through improved customer response times, lower cost to serve, and freeing up capacity. Teams focus on how much time AI creates and whether it’s relieving pressure on the support organization. These signals help validate that the system is working, but they say little about how that capacity is ultimately used.
As deployments mature, measurement starts to reflect a different intent. Instead of stopping at time saved, teams look at where that capacity is reinvested—into higher value customer work and revenue-generating activities. ROI becomes less about relief and more about leverage. I encourage teams to set targets for capacity redeployment and tie them directly to activation, retention, and expansion outcomes.
The report data shows this clearly. Across all maturity stages, the most commonly cited measure of ROI is "time freed up that the support team can use to focus on value-adding activities for customers." But at mature deployment, that signal intensifies, with 73% of teams citing it, compared to 56% at early exploration.
Mature AI deployments reveal clearer ROI: teams report more time freed for value-adding customer work (73% vs 59%) and more hours redirected to revenue-generating tasks (56% vs 34%) than initial rollouts.
What’s also interesting is that 56% of mature teams say freed capacity is being directed toward revenue-generating activities, up from 34% at initial deployment. That’s a powerful indicator that AI is shifting from a cost narrative to a growth narrative.
The result is a shift in economic intent: from measuring what AI saves to demonstrating how the capacity it creates is reinvested to drive growth. As a product leader, I anchor this conversation in outcome-based metrics and clear counterfactuals: what would it have cost to deliver the same experience without AI?
As AI takes on more work, the question moves from “does it save money?” to “how does it change the economics of support?” Legacy support economics were built for linear growth: more customer tickets meant more headcount, more outsourcing, and more software costs. Success was measured through containment—the number of queries that didn’t reach human agents. These models worked when volume and effort were tightly linked, but AI doesn’t scale linearly, and it needs to be evaluated differently.
To sustain AI investment and expand its impact, teams need to move beyond cost-cutting narratives and build a clearer case for business value. When done right, AI goes far beyond improving support efficiency. It rewires the financial model, breaking the link between support costs and revenue growth, and turning support into a contributor to customer activation, retention, and lifetime value. This means treating your AI Agent as a new workforce capability that changes how your support function creates and captures value. Here’s what value looks like in an AI-first model:
Deeper AI integration decouples growth from headcount. This split chart shows support volume surging while team size plateaus, revealing how automation unlocks scale, reduces costs, and makes ROI easier to prove.
Human productivity: Your team focuses on more strategic areas, not the queue.
System improvement: Every resolved query makes the system smarter.
Revenue influence: Support becomes a lever for activation, retention, and growth.
Organizational agility: You scale service without scaling headcount.
Leaders are racing ahead with real AI in support. Explore the 2026 Customer Service Transformation Report to see where deployment is stalling, benchmark your team, and get practical steps to scale automation that delights.
How does this look in practice? Intercom offers a compelling example with Fin. What started as a focused effort to improve their customer support experience has become one of the clearest illustrations of what happens when AI is fully embraced across an organization.
Since 2022, Fin has helped Intercom absorb more than a 300% increase in customer demand while improving the consistency of delivery—including supporting new routes into support for trial customers and website visitors. Today, Fin is involved in 97% of their customers' conversations. Of those, it resolves 83.5% end-to-end, putting their overall automation rate at 81%.
That depth of deployment allowed Intercom to scale service without scaling headcount. Without Fin, they would have needed at least 100 additional support teammates to meet rising demand and service standards.
As Fin took on the majority of day-to-day volume, the human support team shifted toward consultative work—helping customers adopt Fin more deeply, succeed faster, and unlock more value from the platform. Intercom now tracks metrics like “direct revenue generated” and “expansion revenue influenced” to understand the impact of these consultative support activities. This repositioned support from a cost center to an active contributor to long-term growth.
The throughline from The 2026 Customer Service Transformation Report is that deployment depth makes a significant difference. Teams that are investing in deeply integrating AI are reshaping how support scales and contributes to growth. Value becomes clearer as AI takes on more work, and support leaders can articulate that value to the rest of the business.
The gap between these teams and those still in the early stages is widening. A select group of pioneers are setting a new bar for what AI-powered customer service can deliver, and understanding what they’re doing differently is the first step toward closing that gap. If you want to dive deeper into the data and frameworks, you can download the report here: https://www.intercom.com/customer-transformation-report?utm_source=blog&utm_medium=internal&utm_campaign=20260128-report-owned-2026cstransformationreport&utm_content=chapterseries_2
Your team can generate a PRD, summarize an interview, and draft acceptance criteria in minutes. Yet the product still may not ship faster. Customer evidence remains scattered, decisions lose their rationale at handoffs, and nobody knows whether an AI-generated recommendation deserves to be trusted.
An AI-native product development workflow fixes that operating system. It connects evidence, decisions, delivery, and evaluation in one traceable learning loop. The goal is not to produce more documents. It is to shorten the path from a customer signal to a reliable product decision, then carry the result back into the next decision.
Change the unit of work from an artifact to a decision
AI-assisted teams use a model inside an existing process. They write the same documents, hold the same handoffs, and make the same decisions, only with faster drafting. That can save time, but it leaves the fundamental bottlenecks untouched.
An AI-native workflow reorganizes the process around decisions. Every meaningful unit of work should carry enough context for the next person or system to understand what is being decided, why it matters, and what evidence would change the decision.
Use a decision packet with five parts:
Decision: State the exact choice in front of the team. Replace broad assignments such as improve onboarding with a decision such as whether to change the first-session setup flow for a defined customer segment.
Evidence: Link the customer examples, research moments, usage data, and business constraints that support the problem. Preserve the original evidence rather than storing only an AI summary.
Assumptions: Separate what the team knows from what it believes. An assumption should be written so that new evidence can confirm or challenge it.
Success condition: Name the customer or business behavior expected to change. For an experiment, define the hypothesis and, where appropriate, the minimum detectable effect before exposure begins.
Decision state: Record the owner, status, unresolved questions, next test, and reason for the latest change.
The model can retrieve evidence, compress it, identify inconsistencies, draft alternatives, and check whether required fields are missing. A person still owns the interpretation, trade-offs, priority, and release decision. This boundary prevents polished language from being mistaken for product judgment.
Apply a simple test to every AI-generated artifact: what decision will this change? If the answer is unclear, the artifact is probably workflow noise. If the answer is clear, attach the artifact to the decision packet instead of allowing it to become another disconnected document.
Build an evidence spine before adding more automation
Most product workflows fragment evidence before a model ever sees it. Support tickets sit in one system, sales notes in another, interviews in folders, and behavioral data in an analytics platform. A prompt cannot recover relationships that the operating system never preserved.
Normalize incoming evidence while preserving its source identifier, relevant customer or segment context, and access permissions.
De-duplicate repeated signals and cluster related evidence without erasing meaningful differences between customers or use cases.
Retrieve a small set of representative examples for the decision being made. Do not dump the entire evidence store into the model context.
Write the approved decision, its assumptions, and its rationale into durable external state.
Return experiment results, release outcomes, and new qualitative feedback to the same evidence system.
Keep three forms of information distinct. The evidence store contains raw and normalized inputs. Working context contains only the material needed for the current task. The decision log contains approved conclusions, rejected alternatives, owners, and changes. Mixing all three creates stale prompts, contradictory instructions, and summaries that can no longer be audited.
A prioritization recommendation, for example, should link back to representative customer records and the relevant analytics view. A summary without those links is compression, not evidence. When somebody challenges the recommendation, the team should be able to inspect the underlying material without asking the model to reconstruct its reasoning from memory.
This is also where data governance belongs. Decide which systems the workflow may retrieve from, which fields require redaction, who can see sensitive records, and how model outputs will be retained before connecting those systems. Privacy-by-design, cybersecurity, and regulatory controls need to sit alongside the workflow, not appear as a review after customer information has already crossed an inappropriate boundary.
Run one closed loop from discovery to shipped learning
The product trio remains important in an AI-native workflow. Product, design, and engineering use automation to reach the evidence faster and explore more alternatives, while keeping explicit human gates around interpretation, feasibility, customer experience, and risk. Clear handoffs between context design, external memory, and orchestration make those responsibilities easier to see.
For each stage, name the AI job, the human gate, and the durable output. That turns a collection of AI tools into an operating workflow.
Stage
AI accelerates
Human gate
Durable output
Intake and triage
Normalize, de-duplicate, cluster, and retrieve representative customer signals.
Verify that a cluster reflects a real customer problem rather than repeated wording or a noisy channel.
An opportunity record linked to original evidence.
Discovery
Draft interview guides, summarize transcripts, extract entities, and tag moments of friction.
Interpret what the customer meant, identify contradictions, and decide which uncertainty deserves another conversation.
An evidence-backed problem narrative with open questions.
Opportunity sizing
Organize evidence against a driver tree and assemble available inputs about potential impact.
Choose the outcome, inspect data quality, expose assumptions, and make the prioritization trade-off.
A ranked opportunity with decision criteria and explicit assumptions.
Test desirability, usability, feasibility, strategic fit, and the cost of being wrong.
A solution hypothesis, acceptance criteria, and a test plan.
Planning and execution
Break an approved bet into sequenced work, surface dependencies, and check artifacts for missing requirements.
Set scope, choose rollout controls, confirm instrumentation, and approve release readiness.
An instrumented release plan connected to feature flags, CI/CD, and observability.
Iteration
Compare expected and actual outcomes, organize qualitative feedback, and surface anomalies for review.
Decide whether to scale, revise, stop, or collect more evidence.
An updated decision record returned to the evidence spine.
Exit criteria keep each stage honest. Discovery is not complete because the transcripts have been summarized. It is complete enough to move forward when the team can name the customer problem, the supporting evidence, and the uncertainty it intends to resolve next. Solution shaping is not complete because a PRD exists. It is complete when the hypothesis, constraints, acceptance criteria, test method, and required telemetry are clear enough for a responsible decision.
Plan measurement before release. If the team will use an A/B test, write the hypothesis and minimum detectable effect before looking at the result. If controlled experimentation is not appropriate, name the expected behavior change and the qualitative evidence that would support or challenge it. Feature flags provide controlled exposure, while observability helps the team understand why behavior changed rather than merely showing that it changed.
Engineer context, evaluations, and decision rights together
Reliability cannot be added as a final quality check. Every AI transformation can lose evidence, introduce unsupported language, or carry stale assumptions into the next stage. The workflow needs controls at the moment each failure can occur.
Give each task a context contract
One large prompt that tries to perform discovery, prioritization, specification, and planning will accumulate irrelevant material and conflicting instructions. Break the workflow into smaller tasks, each with a compact context contract:
Use a sub-agent when a task benefits from an isolated context or a separate review, such as checking a PRD against approved evidence. Do not add one merely to make the system look agentic. Every additional agent creates another handoff whose inputs, outputs, permissions, and failure behavior must be evaluated.
Build an evaluation harness before scaling the workflow
An evaluation should answer a repeatable question: does this workflow produce an acceptable result on representative work? A few impressive demonstrations do not tell you whether a prompt, retrieval change, or model update made the system more dependable.
Start with real task types your team already performs. Preserve representative inputs, the evidence that should be used, the requirements an acceptable output must satisfy, and known failure conditions. Then run those cases whenever you change the prompt, model, retrieval logic, tool permissions, or output schema.
Evaluate at least these dimensions:
Grounding: Can each important claim be traced to approved evidence?
Fidelity: Did the output preserve material differences, uncertainty, and constraints rather than flattening them into a convenient narrative?
Completeness: Are the fields required for the next decision present?
Decision usefulness: Does the output help a named owner make a specific choice?
Data handling: Did the workflow respect access, redaction, and retention rules?
Format and tool behavior: Did the model follow the schema and use only permitted systems or actions?
Documents generated, prompts executed, and summaries produced are activity measures. They can rise while product decisions become less reliable. Use four layers of measurement instead:
Learning flow: Time from a customer signal to an evidence-backed decision, time spent waiting at handoffs, and rework caused by missing context.
AI quality: Evaluation results by task, unsupported claims found during review, required fields missed, and human corrections before approval.
Customer outcome: The activation, adoption, retention, or other behavior named in the original hypothesis.
Delivery health: Deployment frequency, change failure rate, and the operational signals relevant to the release.
Keep decision rights visible beside those measures. The model may propose a priority, but the accountable product leader approves it. The model may draft a customer interpretation, but the product trio validates it against evidence. The model may prepare a release plan, but engineering owns operational readiness. Feature flags, access controls, and human approval are not signs that the workflow is insufficiently automated. They are what make greater automation responsible.
Log the decision, evidence references, model version, prompt or workflow version, retrieval configuration, evaluation result, and approving owner. Documenting decisions, model versions, and test artifacts makes a nuanced call auditable and gives the team a concrete starting point when quality changes.
Key takeaways: a 30/60/90-day rollout
Do not begin by automating the full product lifecycle. Start with one recurring decision, connect its evidence to its outcome, and prove that the loop can be operated reliably. A practical 30/60/90 sequence expands from the evidence foundation to selected workflows and then into planning and delivery.
Days 1-30: Map the evidence systems used for one recurring product decision. Define the decision packet, access rules, retrieval path, current human gates, and initial evaluation cases. Build the smallest retrieval-first pipeline that can preserve links from a recommendation back to original evidence.
Days 31-60: Pilot continuous discovery and PRD drafting. Keep approval manual, evaluate representative cases, record recurring corrections, and tighten the context contract. Do not expand until the team can identify why an output passed or failed.
Days 61-90: Extend the proven pattern to prioritization and experiment design. Connect approved outputs to planning, CI/CD, feature flags, and observability. Feed release outcomes and customer feedback back into the evidence spine.
By the end of the rollout, you should be able to trace an AI recommendation to customer evidence, reconstruct why a decision changed, detect a quality regression after a workflow update, and compare the expected outcome with what happened after release. If one of those paths is missing, fix it before adding another agent or automating another handoff.
Your next move can be small. Choose one product decision scheduled for this week. Put its evidence, assumptions, success condition, and state into a decision packet. Then follow that packet through discovery, delivery, and the first outcome review. That single trace will reveal where your workflow is genuinely AI-native and where faster drafting is only hiding an old bottleneck.
