I lead product teams building travel and hospitality experiences, and one lesson keeps repeating: companies that measure what matters move faster. Benchmarks turn gut feel into grounded product strategy, making it clear where activation, conversion, and retention are underperforming—and where we can unlock outsized growth.
Discover exclusive data and strategies from our Product Benchmark Report. Compare the travel and hospitality industry’s performance across key product metrics.
When I evaluate a product line, I start with a simple model: attract, convert, delight, and retain. For travel and hospitality specifically, I focus on search-to-book conversion, onboarding completion, first-booking activation rate, time-to-book, average booking value, cancellation rate, support contact rate, DAU/MAU stickiness, repeat booking rate, and long-term retention. These key product metrics reveal friction in discovery and checkout flows, surface pricing and inventory gaps, and quantify loyalty.
From there, I assemble a test-and-learn plan. Using Amplitude analytics to instrument the funnel and Pendo for in-app guides and product tours, my teams design A/B testing with a clear minimum detectable effect (MDE), prioritize hypotheses, and execute rapid, weekly iterations. This is classic product-led growth: reduce cognitive load in onboarding, streamline search and filter UX, clarify policies before payment, and personalize reactivation nudges to improve user activation and retention analysis.
Benchmarks are only as trustworthy as the underlying data. I insist on strong data governance, privacy-by-design practices, and clear event taxonomies so that insights remain reliable across quarters and across markets. That foundation keeps our decisions defensible with stakeholders and regulators while accelerating delivery.
Finally, we translate insights into action with crisp product roadmapping and sprint planning. Cross-functional product trios align OKRs to the biggest benchmark gaps, and we review progress in weekly performance rituals so every experiment ladders up to strategy. This cadence helps teams stay empowered and keeps leadership focused on outcomes, not output.
If you’re building in travel and hospitality, use these benchmarks as your starting line and your ongoing scorecard. Calibrate targets against peers, double down on what moves the needle, and let the data guide bold, customer-centered bets. When teams rally around meaningful metrics, momentum compounds.
Inspired by this post on Amplitude – Perspectives.
Continuous Discovery Habits turns five this year, and I’m celebrating by inviting you to read it with me. Over 135,000 people have bought the book. I’ve seen these habits transform outcomes, reduce rework, and sharpen product strategy in my teams and across the product community, but I also know it’s not easy to sustain the practice—especially when you feel like the lone champion in your organization.
To make it easier and more social, I’m launching the 2026 Continuous Discovery Habits Book Club. We’ll read the book together—one section per month—with discussion questions, practical exercises, and resources that help you actually do the work, not just read about it. Whether you’re picking up the book for the first time or revisiting it, the goal is to build real muscle memory in discovery.
By December, you won’t just understand continuous discovery—you’ll be practicing it.
Each month, I’ll share a reading guide with reflection prompts, exercises you can run solo or with your product trios, and short videos to help you spread the ideas across your team. I’ll monitor comments throughout the year so you can ask for help, share what’s working, and connect with peers—even if you join late.
I’ll also host quarterly live discussion sessions so we can compare notes, push through sticking points, and swap tactics with other empowered product teams. If you want to participate, grab a copy of the book (or dig up your old copy), share the "Spread the Love" videos to get friends and colleagues on board, reserve time to try the team exercises, and register for the community sessions. Let’s do this.
🎖️ This reading guide is brought to you by New Year, New Habit: The 5-Day Customer Interview Challenge. Become a more confident interviewer in less than a week. You’ll conduct one practice interview a day, get personalized and detailed feedback so you know exactly what to improve, and we’ll be giving out daily prizes to the most improved. Join the challenge today.
This Month’s Reading: Introduction; Chapter 1: The What and Why of Continuous Discovery; Chapter 2: A Common Framework for Continuous Discovery. Estimated reading time: ~40 minutes.
These chapters will introduce you to why discovery and delivery are not phases—they happen continuously. You’ll see a clear benchmark for what "continuous discovery" looks like, learn what product trios are and why they’re the foundation for good discovery, and explore six prerequisite mindsets (outcome-oriented, customer-centric, collaborative, visual, experimental, continuous) you’ll need before these habits can stick. You’ll also get the opportunity solution tree—a visual framework for connecting what you’re building to why you’re building it. Need a copy? Grab the book: https://amzn.to/3hGkNYT?ref=producttalk.org
We learn best in community. Use these short videos to share key concepts with teammates and invite them to read along: What is product discovery? https://videos.producttalk.org/videos/799fdbb41e16ebc4f0/what-is-product-discovery?ref=producttalk.org — a quick intro to the key idea behind discovery work. Defining continuous discovery https://videos.producttalk.org/videos/a79fdbba151ee3c72e/defining-continuous-discovery?ref=producttalk.org — a clear benchmark to aspire to. The rhythm of continuous discovery https://videos.producttalk.org/videos/4d9fd5b4111ee0c2c4/the-rhythm-of-continuous-discovery?ref=producttalk.org — the two small research activities you should do every week. The underlying structure of product discovery https://videos.producttalk.org/videos/449fdbb5191fedc4cd/the-underlying-structure-of-product-discovery?ref=producttalk.org — how outcomes, opportunities, and solutions connect. What’s a product trio? https://videos.producttalk.org/videos/a79fdbb31e1be2c12e/whats-a-product-trio?ref=producttalk.org — why cross-functional collaboration matters.
🎖️ This reading guide is brought to you by Just Now Possible, a podcast about how AI products come to life—straight from the builders. If you are being asked to add AI features to your roadmap, you don’t have to start from scratch. Get a head start by hearing how other teams are navigating similar challenges. Find it on YouTube, Apple Podcasts, and Spotify.
When we reflect and discuss what we read, we absorb more and apply it better. This month is about building awareness of where you are today—no judgment. The first step in any change is getting a baseline. Next month, we’ll take small steps to strengthen the habits.
Here are three prompts for individual reflection. 1) Think about a recent product decision your team made. Did you rely more on opinions, data, or customer input? Get specific. 2) Which of the six prerequisite mindsets (outcome-oriented, customer-centric, collaborative, visual, experimental, continuous) is strongest for you personally? Which would require the biggest shift? 3) What’s your reaction to weekly customer touch points? Does this excite you? Scare you? Something else?
And here are three prompts for team discussion. 1) Who on your team is responsible for discovery and delivery? How interconnected are these activities? 2) How does your team currently collaborate cross-functionally? When product, design, and engineering come together, is it to make decisions—or to hand off work? 3) Think of a recent feature your team built. What opportunity did it address? What else could you have built to address that opportunity?
For this introductory month, focus on seeing your current system clearly. In my experience, visibility alone reveals friction and makes the path to change obvious—and measurable.
Exercise: Draw Your Current Discovery Process. Time: 60 minutes. Do this solo first, then compare with your team. Take a blank sheet and draw how your team actually decides what to build. Show where ideas come from, who makes decisions and how, where (if anywhere) customers enter the picture, and how you know if you built the right thing. Then compare drawings with teammates. Where do perceptions differ? What does that say about your shared understanding?
Exercise: Audit Last Week’s Decisions. Time: 30 minutes. Do this solo or with your team. List every product decision your team made last week—big or small. For each decision, note who made it, what information it was based on, and whether customer input was part of the process (and how). Then look for patterns: how many included direct customer input versus assumptions, opinions, or secondhand information?
If you prefer an audio summary of this month’s reading—including the book chapters and the resources below—listen here: Stop Building The Wrong Things Faster (audio summary by NotebookLM): https://www.producttalk.org/content/media/2025/12/January—Stop_Building_The_Wrong_Things_Faster.m4a
Related in-depth guides to go deeper: Product Discovery Basics: Everything You Need to Know: https://www.producttalk.org/product-discovery/ Product Trios: What They Are, Why They Matter, and How to Get Started: https://www.producttalk.org/product-trios/ Opportunity Solution Trees: Visualize Your Discovery to Stay Aligned and Drive Outcomes: https://www.producttalk.org/opportunity-solution-trees/
Other voices worth reading: Product Discovery: Pitfalls and Anti-Patterns by Chris Jones: https://svpg.com/product-discovery-anti-patterns/?ref=producttalk.org Addressing the Challenges of Product Discovery by Saeed Khan: https://medium.com/swlh/the-challenges-of-product-discovery-6ac6109d13a8?ref=producttalk.org Making Product Discovery Work in Small Teams by Sofia Quintero: https://www.chargebee.com/blog/product-discovery/?ref=producttalk.org Product Waste and the ROI of Discovery by Richard Mironov: https://www.mironov.com/waste?ref=producttalk.org
Related course if you want structured practice: Product Discovery Fundamentals – this course walks you through the complete continuous discovery framework with hands-on exercises: https://learn.producttalk.org/cdh-master-class?ref=producttalk.org
Our live discussion schedule for 2026 (sessions are not recorded): Wednesday, March 18, 2026: 9am–10am PDT and 4pm–5pm PDT. Tuesday, June 16, 2026: 9am–10am PDT and 4pm–5pm PDT. Thursday, September 17, 2026: 9am–10am PDT and 4pm–5pm PDT. Wednesday, December 16, 2026: 9am–10am PST and 4pm–5pm PST. Invitations will go out to Supporting Members and CDH Members two weeks beforehand—reserve the time now.
As you work through this month’s material, connect it to your product strategy, outcomes vs output OKRs, and product roadmapping and sprint planning. In my teams, discovery sticks when product trios own the rhythm, weekly customer touch points are normalized, and the opportunity solution tree keeps everyone aligned on outcomes.
I’m thrilled to learn alongside you this year. Grab the book, invite your trio, and let’s build habits that last.
Every planning cycle, I’m asked the same high-stakes question: should we build or buy? In 2026, with generative AI reshaping the software landscape and budgets under scrutiny, the classic calculus needs an upgrade. The right call can accelerate time to value, protect precious engineering capacity, and sharpen competitive differentiation—while the wrong one can quietly inflate total cost of ownership for years.
“Navigate the build vs buy software dilemma, learn how AI is changing the game, and what you should leverage (and when).” That’s been my north star for product strategy this year, and it’s how I guide teams when the pressure is on.
My first principle is simple: build where we differentiate, buy where we need parity. If the capability is central to our value proposition or our defensibility, I’m inclined to build—often with a phased approach that de-risks scope. If it’s a non-differentiating layer (think billing, analytics plumbing, basic CRM integration), I’ll buy to accelerate, then revisit once scale and specialization justify a deeper internal investment.
AI changes the equation on both sides. On the “buy” side, modern platforms now ship agentic AI, fine-tuning options, and robust APIs that let us compose advanced capabilities fast. On the “build” side, AI workflows and toolchains (from code copilots to eval-driven development) compress cycle time, making bespoke solutions more attainable. The trade-off has shifted from pure functionality to questions of AI risk management, model governance, data privacy, and the portability of prompts, embeddings, and training data.
I evaluate decisions across two economic horizons: time to value versus total cost of ownership. Buying often wins the first round—faster deployment, proven reliability, and lower initial lift. But TCO can creep: integration work, per-seat or consumption SaaS pricing, training, vendor-driven roadmap gaps, and the “shadow ops” of maintaining connectors in our CI/CD. Building flips that profile: slower early velocity, higher upfront complexity, but potentially lower long-run costs and tighter fit with our platform scalability goals.
Operational risk matters just as much as features. I look at incident management posture, SRE maturity, SLAs, and DORA metrics to gauge resilience. If a vendor can’t meet our uptime and recovery expectations—or if their roadmap pace mismatches our deployment frequency—we’re effectively renting risk we can’t control. Conversely, if our team can’t realistically support the operational burden, buying is the safer choice.
Security, regulatory compliance, and data governance are non-negotiables. I assess privacy-by-design, data residency, audit logs, role-based access, SOC2/ISO coverage, and threat detection and response. For AI-heavy systems, I add model lineage, red-teaming practices, PII handling, and retention policies. If we can’t verifiably meet our obligations in a build scenario within the launch window, we buy and require clear data exit and portability clauses.
To keep decisions objective, I use a lightweight scorecard across five dimensions: differentiation, urgency/time to value, regulatory/security risk, integration complexity, and AI leverage/portability. We weight criteria with product trios (PM, design, engineering), run discovery spikes, and validate assumptions with stakeholder management up front. A disciplined scorecard curbs recency bias and helps us communicate trade-offs to leadership.
In practice, I favor staged commitments. When uncertainty is high, we buy to learn—ship value quickly, instrument usage, and collect evidence. If adoption proves sticky and integration pain remains moderate, we double down with deeper vendor integration. If we uncover unique needs or cost inflection points, we pivot to a build plan that reuses learnings, data models, and UX patterns from the bought solution to reduce risk.
AI-specific choices deserve their own pass. For example, if we need retrieval-augmented generation, I’ll often buy for the orchestration and observability layer while building our domain-specific retrieval-first pipeline and prompt engineering guardrails. That split gives us speed plus control: we retain our IP and data gravity while tapping best-in-class tooling that evolves with the ecosystem.
Vendor strategy matters as much as technology. I negotiate clear data export, transparent API quotas, sandbox environments for continuous discovery, and price protections for growth. I pressure-test roadmaps, ask for integration references, and align on outcome-based milestones rather than feature checklists. Strong partners welcome this rigor; weak ones stall—another useful signal.
On the build side, I right-size ambition. We target minimum lovable scope, isolate risk in early sprints, and leverage open source where it’s mature and secure. We design for modularity so we can swap components without rewriting the world, and we budget time for in-app guides and product tours to smooth adoption, because user activation is the real finish line.
Here’s the playbook I return to: buy to validate and compress time to value; build to differentiate and reduce long-run TCO; continuously re-evaluate as the AI toolchain and our scale evolve. With a transparent scorecard, a bias for learning, and a clear view of risk, the build vs buy decision becomes less of a leap of faith and more of a repeatable product management capability.
2026 will reward teams that move fast without mortgaging the future. Make the call deliberately, instrument the outcomes, and stay humble—because the best strategy is the one you can adapt as new evidence arrives.
I wanted to cut through the hype and see what’s actually changing inside customer service teams as AI agents like Fin move from pilots to production. So I analyzed 166 interviews with support leaders, managers, and frontline specialists to understand how roles, workflows, and team structures evolve once AI becomes part of everyday work.
The anecdotes were already loud: AI tools are transforming customer support. But the scale, shape, and consistency of that transformation? Less clear. I went to the source—the practitioners living it—to quantify what’s real and what’s next for customer support AI strategy.
Here’s what I gleaned from the data.
TL;DR — What’s changing
AI is reorganizing core CS operations: Nearly every team (≈95%) reported meaningful workflow changes. Triage, routing, translation, and categorization are increasingly automated. Hybrid human+AI systems are taking their place.
Frontline work is changing to AI oversight: Humans now QA, monitor, and test AI outputs. When it comes to handling queries, they step in for nuance, rather than repetition.
Structural change is widespread but uneven across companies: 83% reported new responsibilities or roles. Some built AI pods, while others retained traditional setups.
Tier 1 headcount demand is falling: 28% saw hiring freezes, slowdowns, or natural attrition at Tier 1 level as AI Agents manage more requests and improve operational efficiency.
Skill gaps are widening inside teams: Data literacy, QA, and cross-functional communication are all rising in value. For many companies, long-term role strategy is lagging behind.
Research methodology
The goal of this research is to understand how many customer service teams have changed their roles, responsibilities and ways of working due to adopting AI agents, as well as understanding how these changes manifest within their organizations.
For this study, the data chosen consists of interviews conducted by the research team, either with Intercom customers or prospects. This data was chosen because the focus of the interviews revolved around the individual experience of the participant, which gives a higher chance of information related to role changes to be present.
The data was collected using Snowflake by pulling all interviews stored in gong conducted by a member of the research team from 01-01-2025 to 14-10-2025.
After the data was pulled, a python script was used to clean the conversation corpus for each conversation retrieved. Common English stopwords (e.g. “and”, “very”, “with”, etc.) were removed, as well as all the text associated with a speaker in the conversation that was not the interview participant(s). This was done to reduce the computational power required for the conversation coding, avoid API timeouts and reduce costs.
After the corpus was cleaned, the OpenAI API was employed, alongside a prompt, to code each conversation using closed codes defined in a closed codebook.
The codes used were:
No role change mentioned: No explicit changes to roles, teams, or reporting lines are attributed to AI/Fin.
Role responsibilities changed due to AI/Fin: Duties/ownership moved between humans and AI/Fin, or scope of a role changed because AI/Fin handles tasks.
Team structure/reporting changed due to AI/Fin: Org/team boundaries, team charters, or reporting lines changed due to adopting AI/Fin.
Headcount/hiring impacted due to AI/Fin: Hiring plans, headcount, staffing coverage, or shifts/rotations changed due to AI/Fin.
Workflow/process changed due to AI/Fin: Steps, triage/escalations, routing, or playbooks changed because AI/Fin alters the process.
Other organizational changes due to AI/Fin: Other changes inside the organization due to AI/Fin that don’t involve a change in responsibilities, team structure/reporting lines, headcount or workflow/processes changes.
Data analysis
166 conversations were retrieved. More than 90% of all conversations report some sort of change either in their role, team, or processes due to implementing Fin, or a similar AI product, with only 13 participants reporting no changes.
Across these conversations, each one could have multiple types of change associated with it (M = 2.35, Med = 2, Min = 1, Max = 4, N = 166).
More specifically, after implementing Fin or a similar AI product:
94.58% participants reported having their processes and workflows disrupted
82.53% participants reported seeing their role and responsibilities change
27.71% participants reported changes in company headcount or hiring
6.02% participants reported their team structure or reporting lines changing as a result
Additionally, 16.27% participants reported a change for a different reason from the ones highlighted above (“Other organizational changes due to AI/Fin”).
Sample representativeness
The sample is representative with a confidence level of 90% and a margin of error of ±6.4% (accounting for an overall unknown population size). The individual confidence intervals for each type of change are as follows.
Workflow/process changed due to AI/Fin: 157 (94.6%), 90% CI: 91.7% – 97.5%
Role responsibilities changed due to AI/Fin: 137 (82.5%), 90% CI: 77.7% – 87.4%
Headcount/hiring impacted due to AI/Fin: 46 (27.7%), 90% CI: 22.0% – 33.4%
Other organizational changes due to AI/Fin: 27 (16.3%), 90% CI: 11.6% – 21.0%
No role change mentioned: 13 (7.8%), 90% CI: 4.4% – 11.3%
Team structure/reporting changed due to AI/Fin: 10 (6.0%), 90% CI: 3.0% – 9.1%
Thematic analysis
1) Automation and AI integration replacing manual steps (94.58%). I see AI workflows embedding into every stage of support. Manual triage, routing, translations, and repetitive responses shift to Fin or similar systems, while agents focus on human-in-the-loop oversight.
Agents’ day-to-day work now revolves around monitoring or fine-tuning AI outputs, not replying to the same questions. In many teams, conversations enter Fin first; humans only step in when nuance or exception handling is required. Testing, QA, and rollout practices have matured too—teams track Fin’s accuracy and iterate intentionally.
2) Humans shift to oversight, AI handles execution (82.53%). The role resets are unmistakable. Support agents and managers move from high-volume execution to optimization, configuration, and measurement. New roles emerge—AI specialists, automation managers, Fin owners—while responsibilities migrate toward strategic analysis and quality assurance.
Duties are redistributed: Fin takes on refunds, triage, simple messaging, even parts of the sales process. I’ve watched some careers pivot toward product/ops or AI systems strategy as managers coordinate testing and monitor adoption metrics.
3) Reductions or slower growth due to efficiency gains (27.71%). Efficiency is real. Many teams reduce Tier 1 headcount needs or slow hiring because AI absorbs simpler requests. Others reallocate people to complex work or AI management. A few still expand—adding automation engineers, implementation specialists, or technical AI leads—but not at past growth rates.
The upshot: organizations handle more volume while stabilizing or reducing staffing, especially at the frontline tier.
4) New AI teams, flatter orgs, fewer escalation layers (6.02%). I’m seeing organizational design catch up to the tech. Some companies form dedicated LLM or automation teams. Others flatten hierarchies, design around workflow complexity instead of region, or merge roles. Dedicated escalation layers shrink as Fin routes or resolves more autonomously.
Team design is getting more modular and data-driven, with clearer ownership for configuration, governance, and Agent Analytics.
5) Broader digital transformation and operational modernization (16.27%). Beyond support, companies are modernizing their operating model: automation-first, digital self-service, better data foundations, and new vendor ecosystems. Collaboration patterns between data, ops, CX, and product/engineering are tightening, with a culture of experimentation and continuous improvement taking hold.
How have customer service roles and responsibilities changed due to Fin/AI agent implementation?
Implementing Fin or a similar AI agent profoundly changes how an organization operates, with around 95% of participants reporting some level of change in their processes after implementation. These systems have significantly reshaped the workflows that customer service teams are used to. Tasks once performed manually, such as ticket triage, routing, repetitive responses, and translations are now handled by AI agents.
“This marks a clear transformation in how customer service agents work: moving away from directly resolving customer queries to focusing on more analytical and procedural work”
As a result, customer service agents’ responsibilities have shifted from performing manual tasks to monitoring and fine-tuning the AI agent whenever its output is inaccurate or incomplete. This marks a clear transformation in how customer service agents work: moving away from directly resolving customer queries to focusing on more analytical and procedural work, such as testing, QA, and performance analysis of AI outputs.
Human agents who still handle conversations tend to do so either because the AI agent cannot yet respond adequately, or because of an organizational choice to retain human involvement for sensitive or high-value interactions. Nevertheless, the need for such roles is diminishing. Around 28% of participants reported a reduction in Tier 1 staff or a hiring slowdown or a full hiring freeze, as AI agents increasingly manage simple requests and organizational attention shifts towards improving automation efficiency.
“In some cases, this has led to the creation of specialized AI teams, reorganizations around workflow complexity, or the merging and redefinition of existing roles”
However, this transformation is not uniform across companies. While some roles have disappeared (particularly escalation layers), others have emerged. Many organizations are reallocating existing staff to AI management or hiring new technical profiles such as automation engineers, implementation specialists, and AI leads. In some cases, this has led to the creation of specialized AI teams, reorganizations around workflow complexity, or the merging and redefinition of existing roles.
Around 83% of participants reported changes to their roles or responsibilities following the introduction of Fin or similar AI agents. Specifically, customer service agents who no longer handle basic queries now focus on managing AI performance, reviewing Fin tasks and improving automation outputs. Managers oversee AI evaluation and implementation, coordinate testing, and monitor AI metrics such as resolution and involvement rates. In some organizations, new dedicated roles have emerged—AI specialists, automation managers, or Fin owners—reflecting a strategic shift toward automation-first, digital self-service models.
These structural shifts are also cultural. I’m seeing teams embrace experimentation, versioning, and eval-driven development while deepening collaboration with data, operations, and product/engineering. The move from outcomes vs output OKRs is palpable: leaders are measuring containment, deflection, CSAT, and time-to-resolution with new rigor.
Overall, a widespread transformation is underway. Roles are broadening, responsibilities are diversifying, and cross-functional collaboration is becoming the norm. Given the pace of gen ai improvement and the rise of agentic AI patterns, I expect these shifts to intensify.
This evolution raises two important questions
Firstly, do customer service agents possess the skills required to succeed in these new roles? While they are experts in customer interaction and company policy, their work now demands new competencies in data analysis (e.g. reporting AI agent performance and how it changes over time), quality assurance/debugging (e.g. Fin output testing and versioning), and cross-functional communication (e.g. if help from another team is required, drafting a business case to justify the resources required could be needed).
Secondly, what long-term strategies are companies adopting to support these evolving roles? Some are reorganizing entirely around automation, while others retain traditional structures. For those undergoing transformation, it remains unclear whether these changes are part of a deliberate strategic plan aimed at achieving specific performance outcomes, or the result of experimentation without defined goals.
Ultimately, Fin’s success— and of AI in customer service more broadly— depends not only on the technology itself but on the people and strategies that shape its use. In my experience, the winners invest early in data literacy, robust QA, clear ownership, and governance; they align product, ops, and CX around a shared AI roadmap; and they measure what matters with disciplined Agent Analytics. That’s how you turn AI workflows into durable customer and business outcomes.
Every week, I watch the cybersecurity landscape shift under our feet. As a VP of Product Management, I’m responsible for building secure, resilient products—and that means understanding how artificial intelligence is transforming the way IT teams defend, respond, and even anticipate attacks.
Learn the ways in which AI is transforming both cybersecurity offense and defense for IT teams.
First, AI supercharges threat detection and prevention. Pattern-recognition models now sift through endpoint telemetry, identity signals, and network flows to surface anomalies in near real time. In practice, that means fewer false positives, faster prioritization, and earlier containment. We’re pairing behavioral analytics with enrichment from our SIEM/EDR stack so analysts get a ranked, explainable view of risk instead of a noisy alert queue—directly improving mean time to detect and laying the groundwork for scalable threat detection and response.
Second, AI accelerates incident response. We’ve embedded LLM-powered copilots into our SOC workflows to summarize alerts, propose next-best actions, and auto-generate draft remediation steps from playbooks. Orchestration then executes routine tasks—isolating endpoints, rotating credentials, updating tickets—while keeping a human-in-the-loop for approvals. To keep this safe, we use privacy-by-design principles, a retrieval-first pipeline for authoritative playbook content, and eval-driven development to measure precision/recall on suggested actions. The result is meaningful reduction in mean time to recover and more consistent incident management.
Third, the offense is getting smarter—and we need to be honest about it. Adversaries use gen AI to craft targeted spear-phishing, deepfake executive voice notes, and polymorphic malware that evades signature-based tools. We counter by red-teaming with AI, deploying deception tech to waste attacker cycles, and hardening identity as the new perimeter (MFA, conditional access, continuous risk scoring). Education matters, too: when employees see how convincing AI-generated lures have become, phishing reports spike and successful compromise rates drop.
None of this works without strong governance. We treat AI like any high-impact capability: rigorous data governance, model access controls, and AI risk management across the lifecycle. We log model prompts and outputs, restrict sensitive data via contextual policies, and continuously test for drift and bias. This is as much an IT leadership challenge as it is a technical one—clear ownership, well-defined runbooks, and regular tabletop exercises make the difference between resilience and chaos.
If you’re getting started, I recommend a focused 90-day plan: identify one high-signal detection use case, one response playbook ripe for automation, and one employee risk area (usually phishing) for immediate uplift. Instrument everything—latency, precision/recall, MTTR—and iterate with a cross-functional group spanning security engineering, SRE, and product management leadership. With disciplined AI strategy and guardrails in place, you can move faster, reduce noise, and stay ahead of adversaries without compromising data or trust.
Every quarter, I revisit the same three questions: Are we accelerating adoption, lowering cost-to-serve, and managing risk without slowing the roadmap? Tools that help me answer all three with clarity earn a place in my stack. That’s why the concept behind Pendo’s Agent Analytics resonates so strongly—it gives product leaders a way to see, in one view, how users engage with AI-powered assistants, in-app guides, and core workflows, and how those behaviors translate into product-led growth.
Increase revenue, cut costs, and reduce risk with Pendo’s Software Experience Management platform. Optimize the entire software experience to drive adoption and improve engagement.
In practice, Agent Analytics functions as a unified analytics platform for the modern product team. I can observe how users interact with agents and nudges inside the product, connect those interactions to user activation and retention analysis, and prioritize improvements that deliver measurable outcomes. The result is fewer blind spots across the journey and a tighter feedback loop between discovery and delivery.
The real value shows up when I pair analytics with targeted interventions. For example, I’ll instrument critical paths, baseline activation, then use in-app guides to remove friction at the exact moment users need help. I incorporate A/B testing and continuous discovery to validate which prompts, pathways, or workflows actually move the needle. With a clean view of adoption, engagement, and time-to-value, my team can double down on what works and retire what doesn’t—faster.
Risk reduction is equally important. With clear behavioral signals, I can spot confusing prompts, unhelpful agent responses, or unexpected drop-offs before they scale into churn or support volume. That visibility informs our product strategy, aligns stakeholders on trade-offs, and keeps our governance tight without stifling innovation—especially critical as AI Strategy becomes part of everyday product decisions.
If you’re weighing whether Agent Analytics deserves a place in your toolkit, consider this: better instrumentation yields better decisions. When you unify guide interactions, agent engagement, and core product usage, you can attribute uplift more precisely, forecast impact with greater confidence, and operationalize product-led growth. That’s how we increase adoption, cut unnecessary cost, and de-risk the roadmap—while building experiences customers actually love.
I’ve never seen great products emerge from a one-sided mindset. Inside-out thinking (strategy-first) and outside-in thinking (customer-first) aren’t rivals—they’re a flywheel. When I weave product vision and defensible differentiation together with real customer signals and behavioral data, adoption climbs, engagement deepens, and the roadmap becomes a catalyst for growth rather than a list of features.
For clarity: inside-out anchors on product strategy, value proposition, and the unique capabilities only we can deliver. Outside-in centers on continuous discovery, user research, and telemetry that reveals what customers actually do—not just what they say. At HighLevel, we pair these perspectives in every planning cycle so we’re bold in direction and grounded in evidence.
Increase revenue, cut costs, and reduce risk with Pendo’s Software Experience Management platform. Optimize the entire software experience to drive adoption and improve engagement.
That promise captures why the blend matters. Product-led growth lives or dies on moments like activation, time-to-first-value, and day-30 retention. Inside-out thinking ensures we’re building toward a compelling vision; outside-in thinking ensures users can discover, adopt, and realize value through clear onboarding, in-app guides, and contextual product tours.
Here’s how I apply it in practice. We start by articulating the smallest, sharpest version of our strategy—who we serve, the jobs we must win, and the non-negotiable outcomes. Then we pressure-test that thesis with continuous discovery: call snippets, funnel analysis, pathing, and retention analysis by cohort. When friction shows up in onboarding or early feature adoption, we deploy targeted in-app guides and tours to accelerate user activation without bloating the product or training costs.
A simple operating rhythm keeps the balance: begin each quarter with outcomes vs output OKRs tied to adoption and retention; instrument flows to expose drop-offs; ship iterative improvements; and reinforce them with just-in-time guidance. We use outside-in signals to sequence what we tackle next, and inside-out conviction to avoid chasing noise. The result is faster learning cycles and fewer expensive reworks.
Measurement closes the loop. I track activation rate, time-to-first-value, engagement with the few behaviors that predict renewal, and the impact of each guide or tour on completion rates. When we see lift, we codify the pattern; when we don’t, we prune and refocus. That evidence-based cadence keeps teams empowered and stakeholders aligned.
Culture makes this sustainable. Empowered product teams own outcomes, not tickets. Stakeholder management becomes easier when decisions are grounded in a clear strategy and transparent evidence from real users. And customers feel the difference when the product teaches itself—meeting them with the right help, in the right moment, without getting in their way.
If you’ve been choosing between inside-out and outside-in, stop. Fuse them. Lead with a crisp product strategy, listen with humility, and operationalize adoption through purposeful onboarding, in-app guides, and product tours. That’s how we compound learning, reduce risk, cut support costs, and accelerate product-led growth.
AI is changing how I build products, not by replacing designers or researchers, but by amplifying the quality and speed of what our product trios can deliver. The real breakthrough isn’t a single tool; it’s the way genAI and traditional methods combine into a tighter discovery–design–delivery loop that shortens time-to-value without sacrificing rigor.
Learn how Pendo’s product design team is using genAI and traditional tools to speed up design and development.
In practice, that’s exactly the pattern I see working across my teams: we treat genAI as part of the AI product toolbox—great for rapid exploration, structured synthesis, and test preparation—while we rely on our proven techniques to validate outcomes. For early-stage concepting, I use prompt engineering to generate multiple storyboard options and interaction flows in minutes, then refine those outputs with our design system for alignment and accessibility. It’s a pragmatic “gen ai for product prototyping” approach that lets us compare more alternatives, faster, with better signal.
On the testing front, AI accelerates everything around A/B testing without diluting statistical discipline. We draft hypotheses, define success metrics, and estimate minimum detectable effect (MDE) with guardrails, then deploy variants via feature flags in CI/CD. That pairing—LLMs for product managers plus eval-driven development—keeps experiments reproducible while boosting deployment frequency. The outcome is fewer opinions, more evidence, and a tighter feedback loop from build to learn.
Research goes from weeks to days when we combine a retrieval-first pipeline for qualitative data with strong data governance. I’ll ingest interview notes, support tickets, and session transcripts to cluster themes, then pressure-test the clusters with live customer calls. Privacy-by-design and AI risk management remain non-negotiable: we redact sensitive fields, constrain context windows, and keep a human-in-the-loop for decisions that affect user experience or compliance.
Where analytics meets adoption, tools like in-app guides and product tours help us translate insights into behavior change. I’ll prototype a flow, auto-generate guidance variants, and run controlled rollouts to target segments, measuring activation and retention analysis in parallel. This is product-led growth in action: discover the friction, design the intervention, instrument the journey, and validate outcomes with unified analytics.
Organizationally, empowered product teams and continuous discovery make the difference. Our product trios work from outcomes vs output OKRs, pairing competitive differentiation with product strategy to keep bets focused. We meet weekly to review experiment readouts, model trade-offs with the Kano Model, and update product roadmapping and sprint planning based on verified learning—never vibes alone.
If you’re getting started, begin with one workflow—say, prototype generation plus structured experiment design—and measure impact across cycle time, experiment throughput, and decision quality. Layer in communities of practice to share prompt patterns, establish eval baselines, and codify what “good” looks like. The companies winning with AI aren’t chasing shiny objects; they’re building a repeatable system that turns curiosity into customer value.
Digital transformation set the foundation, but it’s no longer sufficient. In my work leading product teams, I’ve learned that real competitive advantage now comes from building systems that perceive, learn, and adapt—end to end, across the product lifecycle and the business operating model.
AI transformation goes beyond automation to create adaptive, intelligent organizations. Discover why it’s the next imperative and how to measure success.
Why is this the next imperative? Customers expect intelligent experiences, not just digitized workflows. Markets are shifting faster than roadmaps, and teams need systems that learn in production. For me, AI Strategy starts with a clear value thesis: where can intelligence amplify customer outcomes and compound business impact—whether in onboarding, customer support, or core product differentiation.
Practically, I frame AI transformation as a capability stack: data governance and privacy-by-design at the foundation; a retrieval-first pipeline to ground models in trusted context; agentic AI and AI workflows to orchestrate actions; and eval-driven development to continuously measure quality, safety, and relevance. Layered on top are operating rhythms—outcomes vs output OKRs, rapid experimentation, and incident management—that keep shipping disciplined and responsible.
I start with product discovery. Together with product trios, we target moments where intelligence removes friction or unlocks new value. We translate those opportunities into crisp outcomes (activation, time-to-first-value, resolution rate) and instrument them from day one. In customer support, for example, a customer support ai strategy might blend LLMs for product managers with retrieval-first grounding to deliver accurate, brand-safe answers and escalate seamlessly when needed.
On architecture, I prioritize context window management and robust integrations. CRM integration and event streams from tools like Intercom, HubSpot, Pendo, and a unified analytics platform provide the signals AI needs to adapt in real time. Prompt engineering patterns, guardrails, and privacy-by-design controls ensure responses remain trustworthy and compliant. When applicable, I explore agentic AI to orchestrate multi-step tasks with clear constraints and auditability.
Delivery is where transformation becomes measurable. I combine CI/CD practices with DORA metrics (deployment frequency, lead time, change failure rate, MTTR) to keep iteration fast and safe. On the product side, A/B testing with a minimum detectable effect (MDE) protects rigor, while eval-driven development tracks model accuracy, hallucination rates, and policy adherence before and after release. I tie these to business metrics like user activation, retention analysis, and support resolution time to ensure we’re shipping outcomes, not just output.
Governance is non-negotiable. AI risk management, regulatory compliance, and data governance anchor every phase—from dataset curation to prompt libraries and model routing. Threat detection and response and incident management processes are integrated so we can respond quickly when behavior drifts or new risks emerge.
Transformation also means evolving how teams work. I invest in empowered product teams, continuous discovery, and developer evangelism to spread best practices across domains. We share playbooks, reusable CustomGPT workflows, and an AI product toolbox to scale patterns like retrieval-first pipelines and safe prompt engineering across the portfolio.
The outcome is not just smarter features; it’s a more adaptive business. With clear OKRs, reliable telemetry, and responsible guardrails, AI becomes a force multiplier for product strategy and execution. If you’re moving beyond digital toward intelligence, start small, measure relentlessly, and let outcomes guide the journey.
Experience quality compounds just like code quality. To align teams and accelerate outcomes, I rely on a clear, five-stage software experience maturity model to assess where we are, why we’re there, and how to advance. It turns fuzzy debates into concrete product strategy and reinforces a product-led growth mindset.
Find out where you stand—and what to fix first—with this maturity framework.
Why a five-stage model? It gives product, design, engineering, and go-to-market a shared language for trade-offs, helps us move from opinions to evidence, and ties day-to-day improvements to outcomes vs output OKRs. Instead of spreading effort thin, we sequence the right bets at the right time and build momentum with measurable wins.
Here’s how I apply it in practice. I start with a brief, honest self-assessment across the customer journey: onboarding clarity, user activation moments, in-app guides and product tours, UX writing, support loops, reliability, and analytics coverage. Then I layer in learnings from continuous discovery and product discovery—interviews, usage patterns, and support transcripts—so we see the experience as customers do, not just as we intended.
When it comes to what to fix first, I prioritize prerequisites over polish. If the value proposition isn’t clear, onboarding is confusing, or activation is inconsistent, we address those before adding new features. I instrument the funnel end-to-end, establish a minimum detectable effect (MDE) for A/B testing, and ensure we can answer basic questions about who activates, who retains, and why.
Measurement is non-negotiable. I pair retention analysis and activation metrics with qualitative signals to avoid local maxima. Amplitude analytics helps reveal behavioral patterns, while Pendo and in-app guides close gaps in comprehension and guidance. Intercom and CRM integration with HubSpot connect product signals to account health, so we can see how experience maturity drives revenue and retention.
Operationally, I anchor the roadmap to a small set of experience outcomes, link them to product strategy, and review progress in cadence with leadership. This approach builds product management leadership muscle: sharper stakeholder management, clearer trade-offs, and faster feedback loops. Most importantly, the team sees how each improvement ladders up to a better, more durable user experience.
If you’re mapping your own path across the five stages, start by sizing the gaps that block activation and retention, commit to a few high-leverage fixes, and measure relentlessly. With a shared maturity model, your team gains focus, your customers feel the difference, and your product compounds value with every release.
When agent performance improves, everything else follows: faster resolutions, happier customers, and stronger product adoption. In my role leading product management at HighLevel, I use Pendo Agent Analytics to build a shared, measurable view of how our support motions shape the entire software experience and influence product-led growth.
Increase revenue, cut costs, and reduce risk with Pendo’s Software Experience Management platform. Optimize the entire software experience to drive adoption and improve engagement.
In practice, I connect Agent Analytics with our product strategy by pairing product signals (user activation, onboarding progress, feature usage depth) with operational signals (first-response time, time-to-resolution, and deflection rates). This lets me see how in-app guides, product tours, and contextual tooltips impact outcomes across segments without guesswork.
To separate signal from noise, my team runs small, controlled experiments and targeted A/B tests. For example, we’ll instrument a guide for a complex workflow, then compare cohorts on activation, retention, and support ticket volume. If engagement improves and cost-to-serve drops, we standardize the pattern and scale it.
The real advantage is alignment. By treating analytics as a unified analytics platform that integrates agent activity with product insights, we tie day-to-day support work to our value proposition and roadmap. That transparency sharpens prioritization, accelerates adoption, and creates a clear line of sight from agent coaching to measurable business impact.
For teams getting started, baseline your agent performance metrics, map the key friction points in your user journey, and instrument those moments with precise, helpful in-app guides and product tours. Review outcomes weekly, double down on what reduces effort and drives engagement, and keep refining the loop until adoption and satisfaction compound.
Protecting customer data is non‑negotiable—and it must coexist with our need for precise product insights. In my role, I frame every analytics initiative, Pendo Agent Analytics included, around measurable outcomes and rigorous governance so we can accelerate growth without compromising trust.
Increase revenue, cut costs, and reduce risk with Pendo’s Software Experience Management platform. Optimize the entire software experience to drive adoption and improve engagement.
To make that promise real, I anchor implementation in privacy-by-design. Practically, that means data minimization, purpose limitation, role-based access control, auditable workflows, and clear retention policies. These are the same standards I expect from any unified analytics platform and the operating guardrails my team applies in partnership with security and legal.
On the product side, I focus Agent Analytics on the behaviors that move the needle: adoption, feature engagement, user activation, and time-to-value. Paired with in-app guides, product tours, and thoughtful tooltip design, insights become timely interventions that drive product-led growth—while staying within our data governance boundaries.
Reducing organizational risk demands discipline. I pair analytics rollout with a documented data map, DPIAs where appropriate, vendor risk assessments, and clear incident management protocols. We align with regulatory compliance requirements and integrate with cybersecurity practices for continuous monitoring and threat detection and response.
I track success through business and trust metrics: higher adoption, stronger retention analysis, fewer support tickets, and cost savings from deprecating low-value features—alongside clean audits and consistent adherence to governance standards. The outcome is a tighter feedback loop, smarter roadmap decisions, and sustained customer confidence.
If you’re evaluating Agent Analytics, start with a controls checklist, define the minimum viable telemetry for your KPIs, validate consent flows, and pilot with a narrow audience before you scale. This approach balances velocity with vigilance, ensuring we harness analytics for impact without sacrificing privacy or compliance.
