How do you lead a support team in this new world with AI metrics? That question has been front and center for me as we integrate AI-first customer service tools into our daily operations.
The technology is amazing, but our assumptions and processes for understanding and leveraging AI metrics are very different from traditional support metrics. Our new CX Score is the perfect example.
Two months ago, we launched CX Score – a new way to analyze every single conversation and give you a complete view of your support experience. I was genuinely excited as someone who’s battled with CSAT survey mechanics, teammate exclusion processes for CSAT, and the nagging truth that this is only a small portion of our volume. As we’ve navigated CX Score, we’ve learned lessons that apply broadly to AI metrics. Two takeaways stand out for me.
First, lots of data calls for new processes. One of the first things we noticed was the sheer amount of data. For better or worse, CSAT was a small enough sample size to review every comment – particularly unhappy ones. Our QA team would read and categorize each response, and follow up with customers. Managers would read most comments for their team (~15 in total per manager), and discuss in 1:1s.
But what do you do with 1,600+ reviews across the org? This is the reality of AI metrics, and when you have more data than ever before, the old processes don’t scale. We briefly tried reviewing all unhappy CX ratings. We tried taking a sample, but this felt just as limited as CSAT. We exported the trends and conversation data back into an LLM for analysis, but without in-depth prompting the results were only okay.
What worked was reframing how we use the signal. Because CX Score is great for reviewing trends, we use it to measure week-over-week performance for both Fin and as a team wide KPI for human support. We also use CX Score to review specific targeted areas, like a new hire’s conversations on a certain product area. And we use it to review the customer experience for a group of customers, or to analyze a customer’s entire case history so we can lean in at the customer level.
An isometric blueprint reveals how an AI agent powers modern support—from triage to resolution—linking chat, knowledge, and workflows so teams scale service without losing accuracy, context, or the human touch.
Second, the complexities of AI mean we won’t always know the “why” – and that’s ok! People naturally want to know “the why” – especially support folks. When we started using CX Score, one of the biggest challenges was the team wanting to dig deeper into why a specific score was given. While the score provides a great overview, people wanted a detailed, step-by-step explanation. But LLMs are mostly a “black box” – especially to the everyday person. As AI becomes more and more ingrained in our work, we’ll need to accept not always knowing every detail.
This required a mindset shift for both the wider team and leadership as we moved into a world of AI metrics. We focused on the outcome vs. the process, celebrating positives and highlighting insights and actions previously impossible with only CSAT. We refused to compare to humans, reminding ourselves that many of the unknowns of AI are equally true with humans; even with a large survey, we never know for sure how customers feel. And we acknowledged emotions. Our Ops Manager William would poll the leadership team in our weekly ops meeting, asking, “In one word, how did the CX Score make you feel last week?” That simple ritual gave managers space to surface wins and challenges, and it kept us grounded.
What’s next is continuing to revamp how we work and adjusting our collective mindset for AI tools and metrics. The pros highly outweigh the cons, so I encourage you to jump in and start experimenting with AI metrics, especially where they can augment customer experience, operational cadence, and product feedback loops.
Lastly, this technology is improving very quickly. Just yesterday we added deeper AI explanations and additional attributes to explain the CX Score and aggregate summaries across topics. I’m excited to try it out!
Subscribe to The Ticket here – a bi-weekly LinkedIn newsletter delivering key insights for customer service professionals in this time of mind-blowing change.
In my role leading product management and partnering with forward deployed engineers, I’ve worked with countless companies on AI adoption. Across sizes, budgets, and ambitions, I see the same pattern: teams start with the right intentions and still end up disappointed.
The problem isn’t that AI doesn’t work. The problem is that AI done wrong wastes time, money, and trust — and most teams aren’t set up to vet tools, ask the right questions, or structure implementation for success.
To help teams evaluate and deploy with confidence, I often point leaders to The AI Agent Blueprint. It’s a practical roadmap for a moment when everyone’s trying to figure out what comes next.
In this post, I share the lessons I wish every team had before they started. Whether you’re evaluating a solution like Intercom’s Fin or just exploring what gen AI can do, these are the patterns I rely on to make smart, scalable decisions.
Core concepts to help you vet AI solutions like an expert
Before we get into the common pitfalls, let’s cover a few key concepts. You don’t need to become an engineer to thoroughly evaluate AI Agents, but you do need to understand a few foundational terms. This knowledge will help you:
– Ask sharper questions during demos.
– Spot red flags in vendor pitches.
– Choose scalable, future-proof solutions.
– Guide internal alignment and buy-in.
– Build confidence in your final decision.
A little technical fluency goes a long way. Keep in mind these are just a few of the many terms out there. But here are the ones I’d suggest getting comfortable with today:
Retrieval-Augmented Generation (RAG)
RAG enhances generative AI by pulling in real-time, relevant information from your company’s data sources before generating a response.
Why it matters: Most AI tools claiming to “know your business” only use pre-uploaded or static training data. RAG-based systems dynamically search live sources like help centers, product docs, or internal wikis, making them far more accurate and adaptable (assuming your data hygiene and permissions are in good shape).
Easy way to remember: Think of RAG as an AI assistant with an open-book exam. Instead of relying only on memory (pre-trained data), it searches for the latest, most relevant information before responding. This makes RAG especially useful for AI Agents, customer support systems, and AI-driven search engines, ensuring responses are more accurate and up to date.
Vector search
Vector search enables AI to match by meaning, not just keywords. It converts both the user’s question and your documentation into numerical vectors and retrieves the closest semantic match even when the phrasing differs.
Why it matters: Without vector search, your AI may only work if the user phrases things “just right.” With it, users can speak naturally and still get the correct response.
Easy way to remember it: Vector search is like finding a song by its vibe, not its title. It works by intent, not exact match – essential for intuitive AI experiences.
Agentic AI
Agentic AI goes beyond answering simple questions; it can initiate actions, pursue goals, and carry out multi-step tasks.
Why it matters: Most AI tools today are passive. They only respond when prompted. Agentic AI drives outcomes. For example, Intercom’s Fin is evolving to handle actions like checking order status, triggering refunds, or escalating issues, all without human involvement.
Easy way to remember it: Agentic AI is like a rockstar project manager, not just a note-taker. It doesn’t just reply with information when simple questions are asked. It plans, acts, and follows through to get the job done.
MCP (Model Context Protocol) Server / Client
MCP is an emerging approach for managing AI agents at scale. It involves three core components:
– The model (the AI system itself).
– The context (what data and information it can access).
– The protocol (the rules for how it talks to other tools and data).
Why it matters: As AI gets embedded across your organization, centralized governance becomes critical. MCP ensures agents act within rules, respect permissions, and scale responsibly – without needing to hard-code logic into every use case.
Easy way to remember it: Think of MCP as a control tower for your AI agents. It manages what they know, what data they can use, and what boundaries they stay within.
Understanding concepts matters because they help you ask better questions and spot red flags during vendor evaluations. But understanding terminology alone isn’t enough.
Common mistakes I see teams make
Here are five mistakes I see even well-informed teams make, and how I advise product and support leaders to avoid them.
Mistake #1: Treating all AI tools the same
The AI space is moving fast. It’s a constantly evolving landscape and full of buzzwords, which can create confusion. I often see teams treat “chatbots” and AI Agents as interchangeable, without realizing there’s a massive difference between things like:
– A legacy rules-based bot with generative copy slapped on top.
– A true agentic AI system that takes action, learns from context, and scales with your business.
If you don’t understand core terms like RAG, MCP, or the differences between LLMs and agentic AI, it’s nearly impossible to ask the right questions during your evaluation process. I’ve heard of too many teams buying solutions that are outdated or require heavy upkeep after deployment. Educating your team on the fundamentals gives you the confidence to separate real capability from flashy demos.
Mistake #2: Assuming you can build it in-house
There’s a real cost and complexity of building AI Agents internally – orchestration, retrieval systems, prompt chaining, governance, and more. It’s not just a weekend project. It’s a long-term infrastructure investment. And for most companies, it quickly becomes a distraction rather than a differentiator.
Many teams assume building their own AI Agent will be faster, cheaper, or more flexible than buying. On paper, it sounds reasonable – especially if you’ve got a strong engineering team, access to top-tier models, and a healthy budget. But in practice, that path is much harder than it looks.
I smile writing this because I’ve been there. I’ve built multiple AI apps on nights and weekends. Early wins feel amazing — then reality sets in. Shipping something truly polished, even at tiny scale, demands far more infrastructure, reliability work, and governance than most teams anticipate.
At a company level, those challenges only grow. Building an AI Agent from scratch means committing to:
– Data chunking, embedding, and relevance tuning.
– Prompt chaining, context management, and hallucination reduction.
– Real-time retrieval architecture and RAG pipelines.
– Fine-tuning, model upgrades, and fallback orchestration.
– Security, permissions, audit logs, AI governance… and so much more!
Even well-resourced teams often circle back to buying after burning time, money, and momentum. The true cost of building isn’t just engineering — it’s maintenance and velocity. High-performing teams focus on their differentiators and partner for the rest.
Mistake #3: Betting on the wrong vendor
I often see teams focus too narrowly on slick demos or assume a vendor will “figure it out later.” In a market moving this fast, that’s a risky bet. The result is a tool that can’t keep up, needs constant hand-holding, or becomes too rigid to scale.
The best vendors learn quickly, ship frequently, and keep driving value. When I evaluate, I ask:
– Is the vendor investing meaningfully in AI R&D?
– Does their team have a clear roadmap for improvement?
– Can this system adapt to your workflows without needing engineering support at every step?
– How much ongoing maintenance will be needed?
These questions separate vendors building for tomorrow from those selling yesterday’s technology. You want a partner who’s staying ahead, not catching up.
Mistake #4: Ignoring your internal foundation
Even the best AI Agents need fuel. Your content and systems are the inputs that determine quality. If your help center is outdated, documentation is thin, or APIs are missing, you’ll get “garbage in, garbage out.”
I’ve watched teams buy best-in-class AI and still stall because they hadn’t invested in the inputs that make it powerful:
– A well-structured help center.
– Clear, detailed documentation.
– Internal process visibility (for things like internal AI/copilot).
– Robust APIs.
You don’t need to overhaul everything on day one. But clean, accessible content dramatically improves accuracy, confidence, and resolution rate.
Another misconception is expecting AI to resolve 100% of support conversations immediately. In reality, no AI tool starts at perfection — and your team needs a shared understanding of how resolution rate works to set expectations.
For context, Fin typically resolves over 65% of support questions out of the box, with minimal training needed, and continues to improve month-over-month. What separates great implementations isn’t just where you start; it’s how you optimize. Tightening content, closing automation gaps, and iterating on prompts and retrieval all compound over time.
If you’re not tracking your current resolution rate or don’t know how your vendor defines it, it’s hard to see progress. Establish a baseline, set realistic targets, and measure consistently. Treat resolution rate as a growth metric, not a fixed score.
Final thoughts
The teams that win with AI don’t just adopt tools — they implement future-proof systems that connect knowledge, workflows, and decision-making to drive real business outcomes.
– They don’t build everything from scratch.
– They don’t fall for flashy demos of stale technology.
– They partner with vendors already building what’s next.
If your team is exploring AI — whether you’re starting fresh or rethinking your stack — start with the concepts and lessons here. Use them to evaluate options, align stakeholders, and choose partners who are building what’s next, not just what’s trendy.
And if you want a broader strategic roadmap, The AI Agent Blueprint is a great place to dive deeper. It lays out how to go from launching an AI Agent to building successful systems that scale and drive real business value.
AI isn’t just a trend. It’s a capability your business will depend on. Done right, it becomes your most powerful teammate.
Rolling out an AI Agent doesn’t just change how your team works – it changes who your team is.
I learned that in the crucible of a fast-moving launch. Before we launched Fin publicly, our Support team became its first alpha/beta tester and we had to move fast. No roadmap. No step-by-step guide. Just a powerful new technology, and a steep learning curve.
That experience is exactly what led us to create The AI Agent Blueprint – a resource we wish we’d had when we were starting out, and one we hope will give other support teams a clearer path forward.
Looking back, I won’t lie and say I was cool, calm, and confident about how to do this – I was nervous as hell. I had no idea how to implement an AI Agent and ensure it resulted in huge cost savings and stellar customer experiences.
We had older machine learning technology available to us (shout out to our first-gen chatbot, Resolution Bot), but as a complex software business, we really only used it for basic FAQs. In all honesty, we still had a way to go – both in using automation more effectively and in making the chatbot experience actually enjoyable for our customers.
So why the urgency?
When ChatGPT burst onto the scene nearly three (!!) years ago, Intercom’s Machine Learning team immediately spotted the opportunity and dived into building the world’s first (and objectively best) AI Customer Service Agent.
Suddenly, we were being asked to pilot this brand new technology with real customers and go all in ASAP. Because we were selling this powerful new functionality, we had to use it ourselves and show it off in the best possible light so customers would want to use it too. #nopressure
There was no playbook, just a lot to figure out. As a product management leader, I had to switch into rigorous product discovery while staying execution-minded.
Steady involvement, rising resolutions. From February to July, teams maintain a high 87–93 involvement range as resolution rates climb from 65 to 82—signaling how AI-driven workflows can boost support efficiency and outcomes.
How do we do a phased rollout, but scale very quickly?
How do we QA Fin’s responses and make continuous improvements?
How will we produce and manage all the content Fin needs?
What will we do about all the outdated content we already have?
What are the success metrics now? Should they be different to original Support KPIs?
Who’s responsible for the success metrics? Who manages this newcomer to our team?
It was daunting. We had to take a brand new technology, figure out how to use it, build a team around it, and move at breakneck speed to implement every new feature that rolled out. It was ambiguous, fast-moving, and a massive lift.
But we got there and the results speak for themselves: Fin is now resolving over 75% of our inbound support volume.
An isometric blueprint reveals how an AI agent powers modern support—from triage to resolution—linking chat, knowledge, and workflows so teams scale service without losing accuracy, context, or the human touch.
That outcome didn’t happen by accident. We embedded forward deployed engineers with Support, treated our AI Agent like a product creator in its own right, and used gen ai for product prototyping to tighten our iteration loops. We prioritized a customer support AI strategy that balanced containment with quality: containment rate, CSAT on AI-resolved conversations, first-response latency, and recontact rates became our core scorecard.
That success led to real change for me and my team: new roles, new responsibilities, and new career paths. I now run a whole new function that didn’t exist before: AI Support. We’ve created new and elevated roles like Conversation Designers and Knowledge Managers. Fin hasn’t just changed how we support customers – it’s transformed the structure of our team and the trajectory of our careers.
And now, we’re helping our customers do the same.
In all transparency, if I hadn’t been this close to the work, I might have waited to see how generative AI played out before committing. I might have waited for a blueprint for how to deploy and scale an AI Agent. I wish I had something like that when we got started, or even later when we had a solid foundation but needed to scale our AI strategy.
How much less scary would it be to implement an AI Agent if something like that existed?
Whether you’re just getting started or already using AI in some way, you’re not early anymore—and you shouldn’t have to figure it all out alone. Strong product management leadership, a clear change plan, and tight feedback loops are what separate experiments from outcomes.
That’s why we created The AI Agent Blueprint – a practical map for launching and scaling AI in support. It brings together everything we’ve learned from our own journey, and from working closely with our customers who are doing the same.
If you’re ready to operationalize gen ai in support, align on the right metrics, and redesign roles for the future, this blueprint will help you move from pilots to pervasive impact with confidence.
Bold, pragmatic bets separate teams that merely deliver from teams that truly accelerate. As a product leader, I’m drawn to decisions that reduce friction, empower engineers, and compound over time. Intercom’s recent investment in a new frontend direction is a standout example of this mindset—and it offers lessons any product, engineering, or design leader can apply.
Over the past two years, Intercom made one of the most significant changes an engineering organization can make: moving its core frontend from Ember to React. That choice fits a clear pattern of high-agency decision making in service of speed, quality, and developer experience.
Back in 2014, Ember was the right call for their main application. Its strong opinions and “batteries-included” approach aligned with a strategy I respect: make big decisions once, enable teams to move fast, and spend energy on customer problems instead of endless architecture debates. The result was scale few achieve—more than two million lines of code and 100,000+ pull requests merged.
I’ve been in the room when constraints outgrow the original bet. By 2023, “local builds stretched beyond 90 seconds,” and they were stuck on older framework versions that blocked adoption of modern build tools. Even with deep community engagement and contributions to the Embroider Initiative, the cost of staying put was compounding. Something had to change.
What I admire is the rigor behind their pivot. They ran workshops, health checks, and set explicit trigger conditions—then honored those triggers. When the evidence crossed the threshold, they chose a new path and framed the work with a clear, galvanizing banner: “The Future of Frontend.” That’s the kind of governance and narrative clarity that de-risks large platform shifts.
React quickly emerged as the right fit—not because of hype, but because it met practical criteria at scale. “React was already a core technology at Intercom (powering Messenger, Help Center, and our marketing site),” backed by a robust ecosystem, strong documentation, and broad familiarity internally and across the industry. Most importantly, it integrates naturally with AI-driven developer tools—a non-negotiable for the next decade of engineering productivity.
Fast forward to today, and the momentum is clear. “React is now the default for new UI development at Intercom.” That single sentence says a lot about organizational alignment and execution readiness.
The outcomes speak for themselves. “Blazing fast feedback loops: React builds in under 10 seconds locally, with sub-1s rebuilds – much faster than our Ember app’s 90+ seconds.” That kind of drop in cycle time unlocks more iteration, tighter designer–engineer collaboration, and faster learning loops.
Speed without joy is a half-win. “Higher developer velocity: Engineers consistently report being faster, happier, and more effective, particularly when paired with AI tools like Cursor, Augment, and Claude Code.” I’ve seen similar effects: once teams feel flow again, quality and ambition both rise.
Adoption at breadth matters as much as depth. “Wider adoption: Since March 2025, 10+ Product teams have shipped React features, contributing over 840 pull requests.” That level of traction signals a platform shift that’s not just technically sound but operationally viable.
The AI-enabled developer experience is the real unlock. “AI synergy: React just “clicks” with modern AI tooling. Designers and engineers are using agents to write code, generate components from Figma, and even build design playgrounds themselves.” That’s the future: product creators working in shared, generative environments where ideas move from Figma to code in minutes.
One engineer captured the productivity gain perfectly: “The work I had predicted would take me a week to achieve took me two days”. That’s not a marginal improvement—that’s a step-change.
This story isn’t just about frameworks; it’s about preparing for a decade where velocity, AI-native workflows, and developer experience determine competitive advantage. The ambition to “double our productivity over the next 12 months” requires removing friction, leaning into AI, and standardizing on tools that compound learning across teams. React is a pragmatic enabler for that journey.
I also appreciate the organizational design behind the change. A small, focused group—Team Frontend Tech—partnered tightly with Product teams to shape the new stack, build a design system, and accelerate adoption. That model creates a high-trust bridge between platform and product, which is essential for landing a migration at scale.
For leaders navigating similar crossroads, the playbook is clear: set explicit trigger conditions, articulate the future state, pick a stack that compounds with AI, and invest in a cross-functional nucleus to shepherd adoption. For engineers and designers, this is an exciting moment—one where your tools finally catch up with your ambition.
The takeaway I’m carrying forward: make the bold call when the evidence is conclusive, optimize for feedback loops and flow, and treat AI as a first-class partner in the creative process. That’s how we keep shipping fast, raise the quality bar, and focus on what really matters—solving meaningful problems for customers.
Over the past 18 months, I’ve watched the ground shift under support leaders. For many support leaders, the world before and after AI feels drastically different—and I feel it too.
Rewind to before Q1 of 2023, and while the details varied, the challenges support leaders faced were largely the same as they had been for decades. Before AI, support leaders were tasked with improving the customer experience with under-resourced teams; finding ways to improve the cost-to-revenue ratio; preventing team attrition (despite managing people with difficult jobs and low compensation); and representing customers’ needs to teams with competing priorities.
Support was expected to operate behind the scenes, often absorbing work from other departments. Despite being essential for customer retention, it was still viewed primarily as a cost center, and leaders rarely had strong executive advocacy. Those conditions sharpened valuable muscles—creativity, scrappiness, and people leadership—but they didn’t prepare most teams to operate in an AI-first world.
Now with AI, the mandate has expanded. The core responsibilities persist, but the “how” has changed. Leaders are suddenly expected to be AI experts, spearhead large AI implementation initiatives, and keep operations rock-solid while the plane is being rebuilt mid-flight.
They’re being asked to step out from behind the scenes to center stage and lead the company in its first large adoption of AI. They’re being asked to regularly communicate with executives who previously had little interest in their initiatives or ideas. They’re being asked to run high-lift, high-impact, cross-functional projects without the infrastructure in place to manage it. They’re also now expected to hit AI performance metrics that an executive heard somewhere were possible—targets that might be unrealistic for the actual use case.
Oh, and if they fail, they’ll likely lose their job. And if they succeed, they could cause job loss for their team members. I’ve felt that tension firsthand: accelerate AI to drive outcomes, while also protecting the humans who make your customer experience exceptional.
It’s tempting to wait for the storm to pass—to delay AI change until someone else takes it over and hope they don’t undo what you’ve built. I’ve seen that playbook, and it rarely ends well.
There’s a better approach: harness the storm’s energy to elevate your customer experience, your team, and your own influence.
Harnessing AI’s momentum
This new era can reduce your support operation to a transactional, robotic experience—or transform it into what you’ve always envisioned. The outcome depends on how you respond to the demand to implement AI. This is one of the most unique opportunities of your career: you will have your executives’ attention, unprecedented access to product and engineering resources, and far less friction persuading stakeholders that change is essential for customers and the business.
With the right plan, you can reframe your team from cost center to value driver, expand services instead of sweating basic metrics, and move from surviving to thriving.
Here are the three areas I encourage every support leader to master.
Become the AI subject matter expert
Start by learning. Understand what is actually possible with AI now, and what may be possible in the near future. Go at least a layer or two deeper than the average person using ChatGPT. Know what it takes to implement more than a glorified answer bot—especially if your goal is end-to-end resolution, not just deflection.
Then anticipate the pitfalls I see most often in AI adoption.
Not digging deep enough with vendors. Demos often look similar and impressive with minimal lift. The truth emerges in a proof of concept. Run multiple trials with different vendors to uncover real capabilities and limitations—and to calibrate what “good” looks like for your environment.
Only finding a technology solution, not a partnership. Many tools can deliver similar outcomes; partners are not interchangeable. Choose a vendor whose values align with yours, who will support your use cases post-sale, who moves at a pace you can absorb, and who is committed for the long haul (not merely positioning for acquisition in a year or two).
Not knowing what good actually looks like. Ask each vendor about AI involvement rate and AI resolution rate. Ask what AI CSAT typically looks like in your industry. Document these answers to build benchmarks and set realistic expectations with executives.
Not learning from others’ mistakes. Many teams have overestimated AI’s impact and underestimated the human resources still required. Some laid off hundreds of support team members—only to rehire later—damaging their brand and wasting resources. Move with purpose and pace, but not so fast that you repeat these mistakes.
Not communicating your plan effectively. Be able to articulate why deflecting 50% of inquiry volume does not equal a 50% headcount reduction. Cite logistics like coverage windows and redundancy for SLAs, growth needs, natural attrition, and all the non-inquiry work your team handles. Practice a concise, compelling rationale for executives.
Create a clear AI plan
Your company is in uncharted territory. Unless you’ve hired a specialist recently, none of your executives have deployed AI in support. That makes you the most qualified person to draw the map and lead the way. Here’s what your plan should include.
1) A vendor evaluation plan. Define how you’ll research providers, who advances from demo to trial, how many you’ll test, and in what timeframe. Establish criteria for what AI must accomplish and the effectiveness and quality metrics you’ll use to evaluate it.
2) Implementation phases. AI is not a “set it and forget it” tool. Because AI touches customers so quickly, mitigate risk with phased rollouts. Phases don’t have to be slow—just deliberate. Sequence by audience, use case, and channel, and publish a clear timeline so cross-functional partners can plan resources.
3) How you’ll measure success. Reuse your evaluation metrics and go deeper. Track AI involvement rate and AI resolution rate (together, your deflection rate). Measure quality through CSAT and CX Scores, and run regular QA. Quantify impact on your support cost-to-revenue ratio—your CFO cares deeply about this.
4) How your team will use reclaimed time. If your AI program frees 20% of capacity, what value will you create? How will you improve the customer experience, drive revenue or retention, and upskill your team? Quantify the upside and set milestones for capability-building and value-added work. If you fail to plan this, you will be pushed to let too many people go.
5) How you’ll report on progress. Communication failures sink AI programs. Align with your executive sponsor on format and cadence, then over-communicate—regularly, clearly, concisely. You can’t afford to under-communicate.
Own the initiative at a higher level
Support leaders are great at taking ownership, often absorbing projects other teams drop. This initiative is different: it’s highly visible and enterprise-critical. Treat it like a flagship product rollout.
Project management. Use a tool your team can execute in and that lets you summarize progress succinctly for executives. Borrow best practices from your product managers and signal early that you’ll be partnering with them. Learn your sponsor’s preferred update style and tailor to it.
Communication. Overcommunicate—with brevity and rhythm. Don’t let a week pass without your sponsor knowing status. For executives, I recommend weekly or bi-weekly updates with a one-line summary, three impact statements, and a link to the plan. For example: “Saved customers 30K waiting hours M/M,” “Improved full resolution time by 30% M/M,” “Next initiative will improve X metric by Y%.”
Showcase your thought leadership. Reference industry benchmarks proactively when you set goals, and reactively when questions arise. Having succinct, data-backed answers that tie to benchmarks signals expertise and builds trust.
The storm is here—what will you do?
The pressure around AI is intensifying and isn’t fading anytime soon. This storm can crush your team as you know it—or become the wind under your wings that elevates your support operation to its maximum potential. The choice is yours: wait and risk cuts, or step up as the support AI expert, form a plan, and transform your team into a value engine. I’ve chosen the latter—and I invite you to do the same.
At Pioneer 2025, Fin 3 was announced as the most capable AI Agent yet for resolving deep, complex queries across every channel. As a VP of Product Management, I’ve been eager to see whether an agent can match concierge-level service at scale—and this is the first time I’ve seen the pieces come together in a way that genuinely raises the bar for customer experience and operational efficiency.
The goal is simple and ambitious: give customer service teams the tools to deliver concierge-level service to every customer, every time. To do that, the team built Fin 3 and invested deeply in the Fin Flywheel—train, test, deploy, and analyze—so the system learns faster, behaves predictably, and performs consistently across channels like Voice, Slack, and Discord.
The evolution here matters. We’ve come a long way since we launched Fin 1 just over two years ago. It was the very first AI Agent for customer service and focused on using your knowledge content to resolve informational queries, enabling it to do all frontline support and free teams to do higher-level work. Then we launched Fin 2. It answered the question of whether AI Agents could deliver human-quality service (it could).
Since we launched Fin 2, its average resolution rate has continued to climb to 66% across our 6,000+ customers. Over 20% of our customers are getting above 80%.
Those numbers are impressive, but they revealed an important truth I’ve seen across many product organizations: resolution rate isn’t the whole story. Answering a quick FAQ in chat isn’t the same as investigating a payment dispute or verifying a refund over the phone. The real measure to optimize is automation rate—the share of overall workload handled end-to-end. Fin 3 is built for that frontier, with a focus on two levers: solving increasingly complex queries and expanding into more channels.
Procedures are the big breakthrough for training. They let teams encode multi-step workflows and nuanced business logic—like troubleshooting login issues, handling return requests, or investigating potential fraud—so Fin can resolve them from start to finish. In practice, that means Fin is trained to follow your standard operating procedures carefully while exercising judgment just like a seasoned teammate.
1. Natural language instructions
Teach Fin the same way you’d train a new teammate. You can copy and paste your existing SOPs straight in (most support teams already have them written up in Google Docs or Notion) and describe how Fin should act using natural language. The editing experience feels familiar and lightweight, so teams can start writing Procedures immediately without needing engineers or special syntax.
2. Deterministic controls
When a Procedure needs more structure or precision, you can layer in deterministic elements. Data connectors let Fin check information or take actions directly in your tools. Conditional steps handle decision points (for example, whether a refund should be approved) so Fin’s behavior is consistent and predictable. And when absolute accuracy is essential, you can add small code snippets that guarantee the same input always produces the same output. You can also add checkpoints where Fin pauses for approval or hands off to a teammate before taking certain actions, keeping sensitive workflows under human control.
3. Fully agentic behavior
Conversations rarely follow a happy path. Procedures are designed so Fin reasons in real time, moves up and down steps, or switches between Procedures without getting stuck. If a customer changes an answer, Fin adapts and continues naturally. The result is a fluid conversation that still follows your process end-to-end.
4. AI Assistant support
AI Assistant helps teams write and maintain Procedures faster. You can start with a brief overview and supporting documents; it drafts an initial version based on what Fin already knows from your knowledge base and past conversations. As you expand, it suggests additional controls or generates boilerplate code for API connectors, lowering the barrier to entry and accelerating iteration.
Together, these elements let Fin reason like a human with the precision of software. Most teams can begin with no-code or low-code Procedures and bring in engineering only for advanced integrations. That balance of power and control is exactly what high-performing support leaders need.
“Support needs natural conversation and control. Procedures optimize for both – agentic where you want it, designed where you need it – rather than a generic agent builder.”
– Chris Dalley, Director of Product Management at Intercom
Of course, adding agentic power requires robust testing. That’s where Simulations come in. Real-world workflows explode into dozens of paths across policy thresholds, customer states, and edge cases. Manual testing won’t scale, so Simulations let you pick any Procedure, choose a user or segment, and run a full, multi-turn simulated conversation from start to finish. You see exactly how Fin reasons and where to refine, then re-run as needed.
AI Assistant is integrated here too. If a Procedure needs an adjustment, it suggests changes you can accept with a click. It also recommends additional Simulations for complex Procedures to ensure coverage. As you create scenarios, you store them in a Simulation library so that when products, policies, or teams change, you can run the entire suite to catch regressions early. This is how you build confidence that Fin behaves exactly as intended while your automation expands.
Channel coverage is equally critical for customer service. Customers expect help wherever they are. Fin already works across more channels than any other AI Agent, and now it extends to Slack and Discord with meaningful upgrades to Voice.
Fin in Slack feels native—threaded replies, proper formatting, and controls to determine when Fin responds versus when a human steps in. If a teammate joins, Fin automatically steps back. Every interaction is logged for reporting and analysis, which matters when you’re tuning automation rate and resolution quality.
Discord support brings the same benefits to communities that increasingly serve as support hubs. Meeting customers where they are is how you compound both satisfaction and efficiency.
Voice has evolved dramatically since launch, and this matters because phone expectations are different. Rather than waiting on hold, navigating IVRs, or getting one-word answers from a brittle bot, customers get immediate, natural conversation. Since we launched Fin Voice, we’ve added much more power and configurability: better guidance, more customization, better testing and deployment, and call transcripts and summaries. These make Voice practical to run at scale.
Voice isn’t just chat with speech. Latency must be low because long pauses feel wrong. Answer shape matters—shorter, chunked replies outperform long paragraphs. Interruptions and endpointing are the norm, so the Agent must detect when to talk and when to listen. And cost pressures are higher on phone, which makes automation even more valuable.
How natural the Agent sounds shapes customer trust. When a voice bot sounds robotic, people assume it’s limited and escalate immediately. Fin avoids that by speaking naturally, pacing correctly, and adjusting tone as it listens. It can detect sentiment directly from audio—laughter, frustration, or urgency—and respond with empathy to keep conversations on track.
“We’ve seen that how natural the Agent sounds signals to people how smart it is. If it sounds robotic, they escalate immediately – especially on phone where issues are more urgent.”
– Peter Bar, Principal Product Manager at Intercom
Performance has improved significantly, with latency down around 30–40% since launch—conversations now feel fluid rather than stop-start. Unlike voice systems that falter against large help centers, Fin handles real-world knowledge bases at scale using the same reasoning engine that powers chat. Fin Voice is multilingual out of the box and can currently answer calls in 28 languages, with configurable voices and greetings. You can tailor how it operates day-to-day, from call start to escalation rules and office-hour routing. Every call is logged automatically in Intercom, complete with a transcript, summary, and outcome, giving your team full visibility to review performance and refine over time.
Practically, this means Fin can take on more of the phone workload—triaging calls, summarizing transcripts, and handing off cleanly when needed—reducing average handle time and freeing agents to focus elsewhere. Because Voice runs on the same foundation as chat, improvements to Fin’s knowledge apply everywhere, creating consistent behavior across channels.
“Customers often say they’re amazed it’s not a real person – Fin Voice sounds natural, responds in context, and doesn’t feel robotic at all.”
With Fin set up to tackle more complex queries across more channels, the next question is measurement—how well is it working, and where should you improve? The Insights product answers this with upgrades to CX Score, Topics Explorer, and AI-powered Suggestions.
CX Score gives a unified view of support quality across interactions. The new CX Score Reasons provide a more representative and transparent picture—was a low score driven by product feedback or answer quality? These attributes are built into reporting for full filtering and segmentation, which is essential for targeted improvements.
Topics Explorer analyzes and organizes every conversation into topics and sub-topics to reveal what’s driving volume and impacting quality. The new Topic Trends report highlights the most important weekly changes—volume spikes, drops in Fin resolution, and emerging issues—so teams can act before customer experience is impacted. You can now curate topics with merge, rename, move, and create controls, then get AI-powered reporting on the areas you care about most.
AI-powered Suggestions close the loop by proposing exact, ready-to-publish updates to your help content based on what your support team is saying. Suggestions now spots duplications and contradictions, learns from rejections to improve future recommendations, provides one-click updates if you use Zendesk or Salesforce, and proposes changes to data, actions, and guidance—not just content. That last capability is especially important because it helps you unlock higher automation on complex queries.
Fin 3 builds on everything learned since the first AI Agent for customer service launched in 2023. It’s trained through Procedures, tested with Simulations, deployed across every major channel including Voice, Slack, and Discord, and measured through richer Insights. All of it adds up to a simple outcome: Fin now does more of the work for you, resolving the complex, time-consuming queries that used to belong only to humans.
Learn more about Fin 3 here: https://fin.ai/fin3 . Some capabilities are available now, with the rest rolling out quickly. From a product leadership perspective, the takeaway is clear—optimize for automation rate, govern with Procedures and Simulations, expand channel coverage, and instrument with Insights. That’s how you deliver concierge-level CX at scale with a single AI Agent.
At Pioneer 2025, we shared our most ambitious goal since we first set out to build Fin. I’ve spent my career building products that remove friction, and this is the boldest, most consequential shift I’ve seen for customer experience in years.
Fin will not be just the world’s best Customer Service Agent. It will be the world’s best Customer Agent, capable of handling the entire customer experience.
We’ll continue to obsess about Fin’s ability to support your customers, but now we’re broadening our focus. Fin will be able to contact your customers for the very first time; hold their hands through consideration and purchase; be there with them at every step; and know everything about their life with your business. That’s the level of continuity, empathy, and performance I expect from a truly unified AI Agent.
It’s a big shift, and it reflects the changing future of customer service and experience. As someone who lives at the intersection of product strategy and operations, I see an incredible opportunity for teams to elevate their impact.
Customer service leaders have been at the forefront of the AI transformation for the past two plus years. As this next evolution plays out, you’ll be uniquely positioned to lead how AI powers the entire customer experience. Your playbooks, data discipline, and operational rigor are the blueprint for what comes next.
You can watch the full Customer Agent keynote from Pioneer 2025 here.
Why we believe in the Customer Agent future comes down to two clear ideas that I’ve witnessed in practice across multiple organizations.
1. Multiple AI Agents will destroy the customer experience
We know that AI isn’t just changing customer service. Other teams like sales, success, and marketing are seeing the potential and starting to adopt it too. But if every function deploys its own Agent, you’ll end up with competing priorities, fragmented context, and inconsistent brand voice—exactly the kind of friction customers notice instantly.
But if all of these teams use their own AI Agent, you’ll end up with a mess of competing Agents that will destroy the customer experience. Each one will have its own priorities and configuration parameters; they won’t talk to each other or share customer information by default, and will likely engage with customers in different ways. This is a trap we need to avoid.
2. A truly exceptional customer experience is finally possible
If we can avoid that trap, we’ll finally be able to provide the type of seamless experience that customers have long expected, and long deserved. The AI Agents of today, like Fin, are capable of handling many different use cases across the entire customer journey: lead qualification, onboarding, support, success, and upsell. That opens the door for the first time to previously unimaginable customer experience; one that’s truly seamless, personal, and concierge-level.
We’ve reached another turning point in AI’s trajectory, and for customer service leaders, the opportunity around the corner is huge. In my own teams, the leaders who lean in now will shape standards for governance, measurement, and ROI across the business.
Customer service has been the proving ground for AI transformation. The systems, strategies, and learnings leaders in this space have accumulated over the last two years can define how AI is adopted by other functions. The keynote made this clear: you have the opportunity to lead how AI is rolled out across your organization, not just in customer service.
You already manage the most complex, high-volume customer interactions; you have rich data on customer needs and behavior; and you know how AI Agents perform in the real world. Those insights will be invaluable as AI scales across your business. The Customer Agent future will elevate the role of the customer service leader and give you the opportunity to lead AI implementation across the entire customer journey.
To achieve this vision of Fin becoming a unified Customer Agent, it will need to evolve from being a task-based system into a true agentic system that uses AI to make decisions and pursue high-level objectives. That shift—from task execution to outcome ownership—is the inflection point I’ve been anticipating.
Roles: Fin will have a range of roles (customer service being one) that it can fluidly move between and blend together. Each role will be deeply trained to be a world-class expert at what it does.
Goals: Fin will also have goals to pursue. You’ll be able to tell it your objectives and priorities (for your customers, company, and revenue) and Fin will pursue them, making appropriate trade-offs between goals as needed.
Memory: Fin will develop memory that persists and grows over the customer lifecycle, building deep context of who the customer is and what they’re trying to achieve. The customer priorities it learns on day one will be considered in year 10.
Knowledge: Fin will accumulate deep knowledge of your business – every product detail, policy, process, your history, and plans – to act on a complete view of your customer.
Interoperability: Fin will interoperate with different tools, systems, and channels.
This system will be able to do much more than answer questions or complete tasks. It will adapt on the fly, learn to get better, and use all the context it has to efficiently guide each customer to great outcomes. That’s how we turn AI from a helpful assistant into a dependable operator.
The Customer Agent vision isn’t a far-off idea. Many of our most pioneering customers have started to put Fin to work beyond customer service. They’re using it across the customer journey and want to push it further by applying it to other use cases to create a single, seamless customer experience. I’ve seen this expansion accelerate once leaders prove value in one high-stakes workflow.
Here’s an example: fitness wearables company WHOOP, facing one of their biggest product launches ever, needed a way to handle a very large influx of sales conversations. They used Fin to help manage this surge, and it’s now resolving 84% of their sales conversations.
These early examples show how Fin is already capable of handling multiple use cases across the customer journey. The signal is clear: a unified Customer Agent can drive measurable outcomes in both revenue and retention.
The Customer Agent future will be built from the inside out, starting with the customer service leaders who have been pioneering AI transformation since the very beginning. Your frameworks for quality, escalation, and measurement will set the bar for every other team that follows.
You know how to balance powerful AI with human empathy, and how to translate that into great customer experiences. Other teams will look to you, and you have the ability to lead them through this transformation. In my experience, this is the moment to define standards, instrument the journey, and scale wins deliberately.
The very best brands compete on customer experience. The Customer Agent opens that playing field for the brands that jump first. Those who move now will own the new benchmarks for responsiveness, personalization, and ROI.
We’ll be starting to roll out this new functionality to Fin – roles, goals, memory, knowledge, interoperability, and more – over the coming months. Stay tuned.
I’m focused on the future of the products we’ll build—and how we’ll build them. To see where we’re headed, I find it essential to reflect on the past four decades of product development, from on-prem software to cloud-native platforms, from waterfall delivery to agile and DevOps, and now to Generative AI reshaping how we imagine, design, and ship value.
Those cycles taught us a consistent lesson: when technology shifts, our product practices must evolve with it. We learned to ship smaller, measure better, and iterate faster. Today, we’re at another inflection point where the very process of product discovery, prototyping, and delivery is being augmented by intelligence.
Consider this quote: “Applying AI to the software development process is a major research topic. There is tremendous…”
That unfinished thought captures exactly where we are right now—on the cusp of tremendous potential. I see AI accelerating the full lifecycle: transforming ambiguous problems into testable hypotheses, turning research signals into prototypes within hours, and translating product intent into working code and test suites. Gen AI is becoming a collaborator in product discovery, a catalyst for engineering velocity, and a force multiplier for product management leadership.
When I talk about creating intelligent products, I don’t mean bolting on a chatbot. I mean systems that learn from real usage, adapt to context, and continuously improve outcomes. Intelligent products are instrumented end-to-end: they observe, predict, and personalize—while giving users clear control and transparency. They reduce cognitive load, anticipate needs, and create compounding value over time.
How we create these products must change too. In discovery, I pair structured customer interviews with gen AI summaries to surface patterns quickly. I use gen AI for product prototyping to explore solution spaces before we commit code. Forward deployed engineers work alongside PMs and designers to ship high-signal experiments into real environments, shortening the feedback loop from weeks to days.
Operationally, the playbook includes four foundations. First, a robust data strategy: clean pipelines, privacy by design, and event models that map to user value. Second, a model lifecycle: from prompt engineering and fine-tuning to continuous evaluation and rollback plans. Third, a product discovery cadence that treats experiments as first-class artifacts. Fourth, a design system that includes AI interaction patterns—confidence indicators, explainability, and safe defaults—so experiences feel trustworthy and consistent.
Intelligent products demand responsible guardrails. I define clear acceptance criteria for safety, bias, privacy, and reliability, and I use evaluation harnesses with real-world scenarios to test them. Human-in-the-loop checkpoints remain essential for sensitive decisions. Governance is not a blocker; it’s a quality system that protects users and the business while allowing teams to move fast with confidence.
If you’re getting started, focus your next 90 days on three moves. Identify one high-friction workflow where intelligence can remove toil or accelerate time-to-value. Stand up a lightweight experimentation pipeline that logs outcomes and quality signals by default. And empower a small cross-functional squad—PM, designer, forward deployed engineer—to ship a measurable improvement, not a demo.
The destination is clear: product creators who master intelligent capabilities will deliver outsized impact. The path is practical: blend rigorous product discovery with gen AI acceleration, build trust through transparency and safety, and keep users at the center of every decision. That’s how we’ll create intelligent products that compound value—and why I’m optimistic about what we’ll build next.
When the Internet emerged in the mid-1990’s, it seemed clear to me and many others that we were entering a new era of technology, one where our devices and our servers would all be connected, and where data would largely be stored in the cloud. The Internet was essentially a new platform, and a large…
I remember feeling that same sense of inevitability—and urgency—when I first internalized what that shift meant for product development. Platform changes don’t just add features; they rewrite the rules for product strategy, architecture, and go-to-market. The leaders who moved quickly to reimagine their products for a connected, cloud-first world won. Those who clung to comfortable assumptions faced disruption and denial in equal measure.
Today, I see a parallel inflection point with Generative AI (gen ai). Once again, we’re not just adopting a tool; we’re building on a new platform layer that alters how we discover problems, design solutions, and deliver value. In my role leading product teams, I’ve learned that the most effective response is to pair disciplined product discovery with fast, low-risk experimentation—especially using gen ai for product prototyping—to shorten the path from insight to impact.
Practically, this means forming forward deployed engineers and product creators into tight, outcome-driven squads that run continuous experiments, validate assumptions with real users, and iterate rapidly. It also means establishing product management leadership guardrails: clear problem statements, measurable outcomes, data baselines, and rigorous ethics reviews for AI usage. When we treat gen ai as a platform—not a feature—we unlock new product capabilities while managing risk with intention.
The pattern is consistent across eras: replatforming demands curiosity over certainty, learning over legacy, and speed over perfection. We evaluate where connectivity, cloud, and gen ai can remove friction for customers; we instrument our products to learn faster; and we align cross-functional teams around outcomes rather than output. The organizations that embrace this mindset transform disruption into advantage—while those in denial find themselves reacting from behind.
If you’re leading product in this moment, your edge is how quickly you can learn, prototype, and adapt. Start small, ship frequently, and let evidence—not ego—guide your roadmap. The next breakthrough will come from teams that marry strategic clarity with hands-on discovery, using gen ai to accelerate insight without sacrificing trust, safety, or product quality.
I want to believe that all product people are both ambitious and have high agency. But recently I’ve come to realize that this is not always the case. It pains me to admit that, and my first instinct was that these are not people that I can help. But I’m not quite ready to give up.
Over the years in product management leadership, I’ve learned that ambition and agency are related but distinct. Ambition is the drive for impact, scope, and growth. Agency is the willingness and ability to take ownership, make decisions, and move without waiting for permission. High-performing product cultures cultivate both; when one is missing, impact stalls.
I often see four patterns. High ambition and high agency PMs compound value—they run robust product discovery, shape strategy, and ship meaningful outcomes. High ambition but low agency PMs talk big but stall on execution. High agency but low ambition PMs deliver steadily but rarely move the needle. Low on both signals a deeper mismatch with the product creator mindset.
When I coach for agency, I remove ambiguity and increase ownership. I align the team on clear, outcome-based goals, define decision rights, and increase proximity to customers. I expect PMs to run weekly product discovery—interviews, prototypes, and experiments—so they can act decisively from evidence rather than wait for direction.
When I coach for ambition, I connect work to a compelling mission and measurable business impact. I set expectations for strategic thinking, encourage bigger bets alongside incremental wins, and recognize impact—not just activity. I find that ambition grows when PMs see a direct line from their choices to customer and business outcomes.
A practical routine that works: every week, PMs identify one “agency rep” (a decision they will make without escalation, within agreed guardrails) and one “ambition rep” (a scope-expanding action, like validating a bolder hypothesis or challenging a constraint). These reps build confidence and consistency.
For hiring and development, I look for evidence of both. In interviews, I probe for moments where candidates created momentum from ambiguity (agency) and where they set or raised the bar for impact (ambition). Inside the team, I measure both with simple narratives: how did you reduce uncertainty this week, and how did you expand potential impact? The answers reveal whether we are trending toward a healthier product culture.
If you recognize a gap in yourself or your team, don’t label it as fixed. Treat it as a capability to build. Start small, ship learning weekly, and let those compounding “reps” shift the default from hesitation to action. Ambition focuses our aim; agency pulls the trigger.
I haven’t given up—far from it. With deliberate practice and the right environment, we can nurture product people who dream big and act boldly. That’s the standard I hold for myself, my team, and every product creator committed to meaningful outcomes.
I’m seeing more companies than ever commit to the product model, and the shift is unmistakable. Boards are leaning in, CEOs are being pushed, and the subtext is clear: valuation. That pressure can be a powerful accelerant, but it also introduces a very real dynamic—when pilot teams become the vehicle for transformation, the politics around them can either unlock momentum or quietly poison the well.
In my experience, the politics of pilot teams surface fast: who gets on the team, which domain gets picked, how success is framed, and whether the rest of the organization views the pilots as an elitist “special ops” unit or a path for everyone to follow. If I don’t address these dynamics head-on, I watch pilot teams deliver isolated wins that never translate into a durable product operating model.
Here’s how I approach it. I start by being explicit about purpose: pilot teams exist to de-risk the transformation by proving that empowered product teams, operating on clear outcomes, can deliver business impact in weeks—not quarters. I select problems that are meaningful enough to matter (activation, retention, expansion, cost-to-serve) and bounded enough to win. I staff a cross-functional triad—product manager, product designer, and a senior engineering lead—augmented with forward deployed engineers so the team can learn with customers in real contexts and rapidly ship. The language is deliberate: these are product teams, not projects, and discovery is not optional.
To neutralize the politics, I make the rules visible and fair. Team selection is transparent, criteria-based, and time-boxed. Success measures are defined up front and mapped to valuation drivers—retention, net revenue retention, conversion, and CAC payback—so the board and CEO see line of sight from product outcomes to enterprise value. I secure executive air cover for autonomy and decision rights, and I hold the same governance bar every two weeks: discovery evidence, shipped increments, customer signals, and outcome movement.
Execution-wise, I emphasize product discovery as the engine of speed and learning. The team commits to a tight loop: frame the problem, explore multiple solutions, test with real users, instrument everything, and ship small but frequent increments. We visualize the bets, we narrate the learnings, and we make trade-offs explicit. This cadence builds credibility quickly and reduces the urge to micromanage—because the evidence is always on the table.
The most consequential decision comes after the first 6–12 weeks: what do we scale? I codify the ways of working that made the pilot succeed—team topology, discovery practices, decision rights, metrics, and tooling—and then distribute them through enablement, not edict. I avoid the trap of permanent “hero teams.” Instead, we use the pilots to seed a repeatable product operating model that any team can adopt.
When I present progress, I speak in outcomes and learning, not activity. I show how the pilot teams shortened time-to-insight, increased the pace of value delivery, and built the muscles we’ll rely on at scale. I’m candid about what didn’t work and why; that honesty reduces organizational resistance and builds trust with leadership.
If you’re standing up pilot teams now, start by aligning the board and CEO on the outcomes that matter, pick one or two high-impact domains, staff a truly cross-functional team without hoarding all-star talent, and time-box the effort to about 90 days. Publish a one-page charter, instrument the metrics, and pre-commit to decisions based on thresholds: scale, iterate, or stop. Do this well, and the politics fade into the background while the product model—and your product management leadership—speaks for itself.
As a VP of Product Management at HighLevel, Inc., I wrestle with the build-versus-buy question nearly every week. It’s a timeless dilemma, now intensified by generative AI. As one summary puts it, “One topic that has been around since the beginning of the tech industry, is whether we should build or buy in order to solve some problem? This question applies to traditional IT, as well as to every product team. There are often one or more buy alternatives, but each comes with an associated cost, and…”
My take: build vs buy is not a procurement question—it’s a product strategy decision. The right answer depends on whether the capability creates durable differentiation, how quickly we need to learn, total cost of ownership, and the risks around data, compliance, and vendor lock-in. In practice, I anchor the debate in product discovery: what problem are we solving, for whom, and how will we know we’ve succeeded?
When I choose to build, it’s because the capability is core to our product’s competitive advantage, relies on proprietary data or unique workflows, or demands tight integration across the end-to-end customer journey. In these cases, my team and I accept the higher upfront investment because it compounds into long-term strategic control and faster iteration.
When I choose to buy, it’s because the capability is commoditized, speed-to-market matters more than novelty, or the vendor brings specialized compliance, uptime, or scale that would be expensive to replicate. Buying can be the fastest path to validated learning—especially when we need to unblock a roadmap dependency or de-risk a complex integration.
The AI era changes the calculus but not the fundamentals. With gen ai, we can prototype quickly using off-the-shelf models, then decide if we should converge on a managed service, an open-source stack, or a hybrid. The hidden work is real: evaluation harnesses, prompt governance, data pipelines, monitoring for model drift, and cost controls for inference. These become part of the true total cost of ownership—not just license fees versus engineering hours.
In my teams, I often deploy forward deployed engineers alongside product discovery to co-create solutions with customers. We use gen ai for product prototyping to validate value early, test prompts and retrieval patterns, and stress-test edge cases. If the prototype proves the value, we assess whether to keep the vendor in place or transition to a build for differentiation, control, and margin.
Here’s the practical playbook I use. First, define the outcome and non-negotiables: data privacy, latency, SLAs, and compliance. Second, run rapid experiments to quantify value—speed beats speculation. Third, model TCO across 12–24 months, including staffing, MLOps, eval frameworks, and expected usage growth. Fourth, pressure-test vendor lock-in: portability of prompts, embeddings, and fine-tunes; data ownership; exit paths. Fifth, stage-gate the decision: buy to learn fast, then build (or stay bought) based on evidence.
One recent example: we launched a gen ai capability using a vendor to achieve immediate time-to-value and validate demand. In parallel, we scoped a build option gated by adoption and unit economics. The vendor path gave us customer outcomes within weeks; the build path unlocked deeper integration and margin once the signal was strong. That dual-path strategy reduced risk without slowing us down.
Ultimately, the smartest build-versus-buy choices align with product management leadership principles: focus on customer outcomes, quantify opportunity cost, design for learning, and avoid irreversible commitments when uncertainty is high. In the age of AI, those principles still apply—only faster.
