Tag: AI Strategy

How Product Leaders Should Plan Their 2026 Conference Calendar

Your 2026 conference budget should not begin with a list of famous events. It should begin with a decision your product organization is struggling to make: where AI belongs in the roadmap, why discovery is not changing priorities, how product operations should reduce friction, or which growth problem deserves executive attention.

That shift turns conference planning from a travel exercise into a portfolio of strategic bets. You can choose events for the decisions, relationships, and operating changes they can support – and decline the ones that offer interesting content without a credible path to action.

Decide what each conference must change

There is no universally best product conference. The right choice depends on the uncertainty you need to reduce and what you are prepared to do with the answer.

The real cost is larger than registration and travel. It includes the attendee’s attention, the decisions someone else must cover, the interruption to active work, and the follow-through required after the event. An inexpensive ticket can therefore be a poor investment, while a more demanding trip can be defensible when it helps unblock a consequential product decision.

Before approving a booking, require a short conference thesis with these fields:

Decision: What active product, AI, growth, hiring, or operating-model decision should improve?
Uncertainty: What does the team not know well enough to decide confidently?
External value: Which practitioners, perspectives, or examples are difficult to access inside the company?
Action: What will the attendee recommend if the current hypothesis becomes stronger, weaker, or more conditional?
Owner: Who has the authority to act on what is learned?

For an AI product leader, a useful thesis might be: I need operator evidence that helps us decide whether the next investment belongs in a customer-facing capability, an internal workflow, or the evaluation and governance layer beneath both. That is specific enough to shape session choices and conversations. Learn more about AI is not.

The same standard applies to networking. Meet product leaders is too vague to guide behavior. Compare how product executives assign decision rights between a central AI platform group and embedded product teams gives the attendee a real question, a relevant peer profile, and a reason to follow up.

Key takeaways

Choose a conference for an active decision, not for its reputation alone.
Build a portfolio across strategic priorities instead of sending everyone to similar general-interest events.
Use date collisions to force explicit tradeoffs about content, peer access, and geography.
Send the person closest to the uncertainty, provided that person has enough authority to apply what they learn.
Measure changed decisions and operating behavior, not notes collected or badges scanned.

Build a portfolio around strategic priorities

Conference names are useful screening signals, but they are not proof of agenda quality. Start by matching event themes to company priorities. Then inspect the organizer’s current agenda, speakers, attendee profile, format, and location before committing money or executive time. Verify the date and venue directly before purchasing because event details can change.

The following map helps narrow the field. It is a planning tool, not a ranking.

Strategic priority	2026 events to investigate	Question that should govern selection
AI products and intelligent interfaces	AI Product Summit on April 15 in San Jose, ProductCon AI on August 5 as a virtual event, and ACM IUI from March 23-26 in Paphos	Do you need product operating practices, broad virtual access, or deeper thinking about intelligent user interfaces?
Executive product leadership	Gartner Product Leadership Conference on March 9-10 and Chief Product Officer Summits in New York, Palo Alto, Amsterdam, and San Francisco	Which leadership decision needs calibration with peers who have comparable scope and accountability?
Product operations	Product Operations Summits in New York on March 26-27, Amsterdam on May 12-13, and San Francisco on September 22-23	Are you trying to improve decision flow, planning, discovery infrastructure, tooling, or cross-functional accountability?
Product-led growth	Product-Led Summits across Washington, Austin, New York, Denver, Amsterdam, Seattle, London, San Francisco, Berlin, Sydney, Boston, and Toronto	Which growth problem – activation, adoption, expansion, pricing, or organizational ownership – is important enough to justify attendance?
Discovery, UX, and research capability	ACM CHI, UX360, UXLx, UXDX, UXPA International, uxcon, and Leading Design	Does the team need a new method, stronger leadership practice, or better integration of research into product decisions?

A balanced portfolio can contain different kinds of bets. An anchor event can provide a broad external scan and senior relationships. A specialist event can address a narrow operating problem. A virtual event can provide targeted content without travel. A regional event can deepen a market-specific network. You do not need every type; use only the ones connected to current priorities.

This matters because superficially similar conferences can serve different jobs. A product-led growth event may be useful when the team is redesigning activation or commercial ownership. It is probably redundant when the same attendee recently explored the same questions and no resulting work has reached implementation. Repetition is valuable only when the audience, market, or decision has changed.

AI-themed events deserve an especially strict filter. A prominent AI label does not tell you whether the agenda will help with real product work. Look for evidence that sessions address evaluation, data readiness, workflow adoption, reliability, governance, product economics, and organizational change – not only model demonstrations. If AI hiring is the objective, confirm that the likely participants include the builders or leaders you need to understand, rather than assuming a large technology audience will produce relevant candidate relationships.

Keep some conference capacity uncommitted. Later announcements, agenda changes, and newly urgent company problems can make an event that looked optional more relevant than one selected during annual budgeting. A complete calendar created too early can leave no room for better information.

Use calendar collisions to make sharper choices

The 2026 schedule contains several useful collisions. They are not merely logistical problems. They reveal whether your selection criteria are strong enough to distinguish one event from another.

On March 26 in New York, the Chief Product Officer Summit, Product Operations Summit, and Product-Led Summit occur on the same day, with the latter two continuing through March 27. One attendee cannot meaningfully cover all three. A product executive working through organizational design may favor the leadership audience. A product operations owner rebuilding planning and decision infrastructure has a different reason to be there. A growth leader should not choose either merely because colleagues are attending.

Amsterdam creates a similar choice on May 28-29, when the Chief Product Officer Summit and Product-Led Summit run on the same dates. If both themes matter, split coverage only when each attendee has a distinct brief and a real route to implementation. Dividing the team without separate objectives simply doubles the expense and produces overlapping summaries.

San Francisco offers a cluster rather than a direct collision: the Chief Product Officer Summit takes place on September 17, followed by Product-Led and Product Operations Summits on September 22-23. Combining them may reduce duplicated travel, but it also creates time away between events. The itinerary is justified only if the later event answers a separate, important question and the attendee can use the intervening time productively.

Europe’s June schedule shows why geographic proximity can be misleading. Mind the Product runs in London on June 15-16, Growth Minded Superheroes is in Frankfurt on June 16, UX360 EU is in Berlin on June 23-24, Product-Led Summit is back in London on June 24-25, and Product at Heart follows in Hamburg on June 26. These cities can look like one efficient conference run on a map. In practice, conflicting dates, transfers, context switching, and prolonged absence can overwhelm the incremental learning.

Use these rules when events overlap or cluster:

For a same-city overlap, choose by problem. Split coverage only when the briefs are materially different and both attendees can act afterward.
For a nearby-city cluster, calculate absence as well as travel. Add transfers, working days between events, and recovery time to the decision.
For a cross-region collision, protect the portfolio. Select the event serving the more important company priority, not the one creating more fear of missing out.
For a virtual alternative, distinguish content from access. Choose virtual participation when learning is the main objective; protect in-person travel for relationships or interactions that genuinely require presence.

Also block the attendee’s return capacity before approving the trip. If the calendar is packed with internal meetings immediately after the event, synthesis and follow-up will be displaced by routine work. That turns an expensive learning opportunity into a folder of notes no one uses.

Send the right attendee with an operating brief

The most senior available person is not automatically the right attendee. Neither is the person most eager to travel. Match the attendee to the decision, the conversations required, and the authority needed afterward.

Conference objective	Best-positioned attendee	Authority or support required afterward
Reframe AI product strategy	The product or AI leader accountable for the portfolio decision	Ability to change priorities, commission validation work, or define a new investment thesis
Improve product operations	The leader or operator who owns planning, decision flow, tooling, or product rituals	A sponsor willing to change responsibilities, forums, or operating mechanisms
Strengthen continuous discovery	A product, design, or engineering representative close to active customer and delivery work	A real product area in which to apply the method and partners prepared to participate
Calibrate executive leadership	The product executive who owns the relevant organizational or stakeholder decision	Access to the executive forum where the operating change will be decided
Inform AI hiring	The person accountable for role design, assessment quality, or the hiring decision	Permission to update the role scorecard, sourcing thesis, or interview process

A practitioner sent to an executive event may hear useful ideas but lack access to the people or forums needed to apply them. An executive sent to a method-heavy event may return with broad principles while missing the implementation detail. When the objective spans levels, assign a primary attendee and an internal sponsor instead of assuming one person can represent every perspective.

The operating brief should travel with the attendee. Include:

Outcome statement: The decision or operating change this event should inform.
Question set: The questions that sessions and conversations must help answer.
Current hypothesis: What the team presently believes, including the conditions that might make it wrong.
Conversation map: Relevant peer operators, complementary functions, speakers, customers, partners, or prospective hires to look for.
Agenda rules: Sessions that directly answer the brief, useful alternatives, and content that can be skipped without regret.
Capture format: Claim, evidence, context, transferability, open question, and follow-up.
Coverage plan: Decisions delegated during the attendee’s absence and the escalation path for anything that cannot wait.
Return forum: The operating meeting where recommendations will be considered, not merely presented.

Networking becomes easier when the attendee has a real problem to discuss. A useful introduction contains four things: your role, the situation you are working through, the precise question, and a modest request to compare approaches. That gives the other person something concrete to respond to. A generic request to connect transfers all the work to them.

Do not optimize for the number of conversations. Optimize for relevance and continuity. A smaller set of exchanges tied to active work is more useful than a long contact list with no reason for another interaction. Before leaving the event, record why each follow-up matters and what question should move forward.

Measure return through decisions and behavior

Conference return is often reduced to attendance, notes, leads, or an internal presentation because those outputs are easy to count. None proves that the company made a better decision or changed how it operates.

Use a decision ledger instead. Before the event, record the active decision, current hypothesis, unresolved evidence, owner, and forum where action can be authorized. After the event, update the same record with the strongest signal, its origin, the context in which it appeared to work, the resulting confidence change, and the next action.

Evidence quality matters. A vendor claim, a conference-stage success story, a private conversation with an operator, and a pattern repeated across unrelated practitioners should not receive equal weight. The attendee should label which kind of signal they captured and identify what still needs internal validation. Conference learning can shape a test or decision; it should not bypass product judgment.

Return channel	Evidence that counts	Weak proxy
Decision quality	A documented product, AI, growth, or operating decision was changed, accelerated, narrowed, or deliberately held pending better evidence	A polished event summary
Execution	An experiment, discovery activity, hiring change, or operating practice has an owner and a checkpoint	General enthusiasm about trying new ideas
Network	A relevant relationship continues around an active problem, with a clear reason for follow-up	Contacts or badge scans collected
Talent	The team improves a role thesis, evaluation approach, market map, or ongoing candidate relationship	A stack of resumes without fit assessment
Knowledge transfer	Colleagues can apply the insight to current work and understand the conditions under which it may fail	Slides placed in a shared folder

Not every conference needs direct revenue attribution. Executive calibration, hiring insight, and discovery capability can create value indirectly. Forcing a speculative revenue number onto those outcomes creates false precision. It is still reasonable to demand a visible chain from attendance to evidence, from evidence to a decision, and from the decision to owned work.

Cancel or reassign attendance when that chain is missing. A booked event should be reconsidered if the final agenda no longer matches the priority, the attendee cannot articulate a decision thesis, the same questions were recently explored elsewhere, or no one has capacity to act on the result. Sunk planning effort is not a reason to spend more time and money.

Open your planning calendar with one real company priority, then review the published 2026 product conference dates. Shortlist the event whose audience and agenda best match the unresolved decision. Assign the decision owner, write the operating brief, protect the return forum, and only then approve the booking. If an event cannot survive that sequence, decline it and preserve the capacity for one that can.

References

Product Talk – Plan Your 2026 Product Conference Calendar: Top Events, Locations, and Insider Tips

December 17, 2025

2026 Support Capacity Playbook: Bold AI Automation, Smarter Staffing, Zero‑Surprise SLAs

Capacity planning has always been a high-stakes exercise in customer service, and when you miss, the signal shows up fast in backlogs and SLAs. I’ve lived that pressure across multiple cycles, and 2026 will reward teams that plan differently. AI fundamentally changes capacity planning because it changes the work. It resolves the bulk of your volume, speeds up execution, and elevates the complexity and value of what humans handle. The consequence is simple: planning models must evolve. This is the final installment in my 2026 customer service planning series, and I’m focusing on the tension every leader feels right now—be ambitious about automation, but avoid the trap of understaffing if your assumptions don’t hold. My goal is to share how AI changes the logic of capacity planning, what I’ve learned implementing these practices with my team and with customers, and the common traps to avoid. Traditional planning rests on relatively stable assumptions: volume grows predictably, work types stay consistent, handle times don’t swing dramatically, and productivity improves slowly with better tools and training. In an AI-first model, none of that is guaranteed, and the fundamentals flip. The mix of work changes as AI absorbs a growing share of simpler conversations, leaving humans with deeper, more time-consuming issues that demand human-to-human connection. Demand can actually increase when you remove friction, so AI can both resolve more and attract more volume. Human time splits differently as teammates solve customer problems and also review AI behavior, give feedback, improve content, and support system-level work. Performance becomes dynamic, not fixed—automation rate isn’t a one-time number; it can rise with care and fall with neglect. If you plan for 2026 using a pre-AI model—assuming similar productivity, similar work mix, and a linear relationship between volume and headcount—you will underestimate what it now takes to run a high-performing support organization. There are many metrics you can track, but the one to put at the center is automation rate (AI Agent involvement rate × AI Agent resolution rate). This single construct tells me what share of total volume AI actually resolves, how much work remains for humans, how much additional demand humans can absorb, and how ambitious I can be with headcount. Early in the journey, I prioritize raising involvement—getting the AI involved in more conversations. Once involvement is high, I shift to resolution on the hardest remaining work, where each additional 1% of automation can represent several people’s worth of capacity. In my 2026 plans, automation rate sits alongside projected inbound volume, average “output” per person for the more complex work that remains, and occupancy—how much time is allocated to customer-facing interactions versus operational and strategic work. Together, those inputs give a realistic picture of how many people you need and where they should spend their time. First, plan boldly on automation, but match it with investment. I do not cap automation assumptions at 40–50% “because AI is new.” Many teams are already modeling 60%, 70%, even 80%+ for 2026—when they invest in AI ownership and content. The investment is non-negotiable: named ownership for AI performance (AI ops, knowledge management, conversation design), clear automation targets by work type (e.g., informational vs. personalized vs. actions vs. deep troubleshooting), realistic expectations for what’s easy to automate and what’s not, and a concrete plan to raise automation over time in monthly or quarterly steps rather than a single jump. To decide where to invest first, I dig into the data. I start with the biggest volume drivers, separate content-led issues from those dependent on data or complex procedures, assume higher resolution potential for content-led topics once the knowledge base is in shape, and set more modest initial resolution expectations for system-dependent flows. Then I stair-step improvements as the systems, data contracts, and workflows mature. In short, bold automation goals only work when paired with the team structure, content, and systems required to reach them—and the discipline to iterate. Second, expect human “output” per person to go down. That’s a mindset shift. Historically, we assumed individual productivity would stay flat or tick up as tools improved. In an AI-first model, humans handle fewer conversations but more complex, cross-functional issues—and create more value despite lower case counts. I model a lower “cases closed per person” than prior-year baselines, explicitly assume the remaining work is more complex and time-consuming, and redefine productivity to include system-level work like AI Agent improvements, content updates, and policy or workflow change management. I also report “capacity created” from automation alongside human outputs, so leadership sees the full picture. Third, rethink occupancy: more time off the queues, on higher-value work. Traditional occupancy splits time between inbox and training, meetings, and breaks. Now there’s an expanding “out-of-inbox” portfolio that directly affects AI performance and overall capacity: reviewing AI-handled conversations, improving AI Agent triaging and handovers, contributing to content and procedures, feeding insights to product and engineering, and supporting system changes that reduce future volume. I set lower inbox occupancy targets than before and make the rationale explicit. People aren’t working less—they’re working differently. In planning, I assume more time spent on improvement and system work, make it visible (for example, X% in inbox and Y% on AI and system improvement), and treat this as critical, not a “nice to have.” If you don’t proactively allocate it, it won’t happen—and your automation and performance targets will suffer. Fourth, work with the finance team early, and treat your plan as a set of assumptions. Capacity planning with AI is a set of bets across automation rate, human output, demand growth, occupancy, and where surplus capacity (if any) goes. I bring finance in early, show that the plan is dynamic and directly tied to AI performance, and label every lever as an assumption with ranges. I commit to a quarterly review cadence with finance to compare assumptions versus reality and adjust headcount, targets, and investment as needed. The risks are real: if automation grows slower than expected and you stop backfilling too early, you’ll be understaffed for months. Hiring and onboarding take time, so course-correcting late creates strain. If you do produce surplus capacity, have a clear strategy to reallocate those teammates to higher-value work—improving systems, feeding insights back to product, supporting new channels, and driving proactive CX—rather than defaulting to reductions. I also set explicit guardrails—if automation rate misses by five points for two consecutive months, we pause planned reductions and revisit hiring gates. If it over-performs, we shift people into backlog eradication, content upgrades, or proactive outreach, so we bank compounding value. To set your team up for success in 2026, anchor your plan on automation rate, be honest that humans will handle fewer but harder conversations, and protect time for system improvements. Partner early and often with finance, avoid shrinking too fast, and design a plan for surplus capacity so you’re never caught flat-footed. If AI is going to handle the majority of your customer conversations, your plan has to be designed to help it do that well and to keep your team set up for meaningful, sustainable work. A 2026 plan built on adaptable assumptions—not fixed predictions—will hold up as your work, your systems, and your customers’ expectations continue to change. If you’d like future editions like this, subscribe and stay close—I’ll keep sharing what’s working, what isn’t, and how to tune your customer support AI strategy in real time.

Inspired by this post on The Intercom Blog.

December 16, 2025
AI in Product Design: My Proven Playbook, Real Use Cases, and the Tools That Win Faster

In product design, AI has shifted from novelty to non-negotiable. I’ve watched teams accelerate discovery, compress prototyping cycles, and turn ambiguous ideas into validated experiences faster than ever—without sacrificing quality or customer trust.

AI in product design has quickly moved from new to necessary. Here are the AI product design tools and approaches you need to stay relevant in this decade.

From my vantage point leading product teams, “necessary” means AI is woven throughout the product lifecycle—discovery, prioritization, prototyping, validation, and iteration—not bolted on. The goal isn’t to chase hype; it’s to build durable advantage with clear AI Strategy, disciplined execution, and measurable outcomes.

First, anchor the work in strategy. Tie every AI initiative to a specific customer problem and value proposition, then express that linkage with outcomes vs output OKRs. This keeps teams focused on real impact and avoids feature-chasing. It also sharpens product positioning and clarifies where AI can deliver competitive differentiation versus simple points of parity.

Second, upgrade discovery. I rely on AI workflows to synthesize interviews, cluster themes, and surface insights at scale. A retrieval-first pipeline—grounding models in our own data—improves factuality and reduces hallucinations. Combine this with strong data governance and privacy-by-design so insights are trustworthy and compliant from day one.

Third, make quality measurable. Adopt eval-driven development: define evaluation sets and acceptance thresholds that reflect real user tasks before you ship. Pair that with A/B testing and minimum detectable effect (MDE) discipline, so you learn quickly and confidently. Add safety guardrails (red-teaming prompts, content filters, and bias checks) to manage AI risk without slowing the pace.

Fourth, enable empowered product teams. Product trios (PM, design, engineering) should co-create prompts, prototypes, and evaluation criteria. Give designers and PMs practical tools—LLMs for product managers, structured prompt templates, and reusable components—so AI-augmented work becomes the default, not a special project.

Where does AI shine in product design today? Concept exploration and market scans, turning fuzzy opportunity spaces into crisp problem statements. Rapid wireframes and interaction ideas, using gen ai for product prototyping to explore multiple design directions in minutes. UX writing that adapts tone and reduces friction across onboarding, tooltip design, and microcopy.

It also excels at guided experiences. I’ve seen strong lifts in user activation when we pair in-app guides and product tours with context-aware suggestions. For support and education use cases, a retrieval-grounded assistant can deflect tickets, shorten time-to-value, and reinforce the product’s value proposition at the exact moment a user needs help.

Voice is another frontier. A well-scoped voice AI agent can accelerate complex workflows (think data entry or multi-step configurations) when hands-free is faster or more intuitive. Just be intentional about when agentic AI adds net value versus when a simple UI tweak would do.

On the tooling side, my AI product toolbox is pragmatic and modular. For analytics and learning loops, Amplitude analytics and Pendo help quantify behavior changes and retention analysis. For in-product engagement and feedback routing, Intercom and HubSpot integrate cleanly with LLM-driven tagging and summarization. For ideation and automation, I use a ChatGPT connector and Claude Code for quick scripts, data wrangling, and prompt experiments. The constant: a retrieval-first pipeline that grounds models in approved knowledge and maintains context window management at scale.

Risk management is built in, not bolted on. Set clear AI risk management policies, catalog model and data dependencies, and document decisions. Align with regulatory compliance requirements early, and keep an audit trail of prompts, datasets, and eval results. That’s how you move fast without breaking trust.

If you’re getting started, begin small: pick one high-friction workflow, add a retrieval-grounded copilot, and measure the lift. Use the results to inform product roadmapping and sprint planning, then scale to adjacent use cases. With disciplined discovery, sharp evaluation, and the right tooling, AI becomes a force multiplier for product teams and a clear win for customers.

Inspired by this post on Product School.

December 15, 2025
From Concierge to AI Marketing Engine: Inside Mowie’s Document Hierarchy Playbook

I’m constantly asked by SMB owners: What if your small business could have a full marketing team—automated content calendars, customer segmentation, and channel-specific posts—without the headcount? That question is no longer hypothetical; it’s precisely the promise behind Mowie, and the way they got there is a masterclass in practical AI product development.

I recently listened to Chris O'Connor (CEO) and Jessica Valenzuela (Co-Founder) of Mowie, an AI marketing platform built for small and medium-sized businesses in restaurants, retail, and e-commerce. Their story starts with a concierge marketing service—doing the work by hand for overwhelmed owners—and evolves into a fully automated AI product.

They walk through their "document hierarchy" approach: how Mowie crawls the web to build a "dossier" about each business, infers customer segments and marketing pillars, and generates quarterly content calendars with channel-specific posts. As a product leader, this is the kind of retrieval-first pipeline that consistently outperforms naive prompt chaining because it builds durable context before generation.

They also unpack the technical challenges of structuring unstructured data and the evolution from rigid schemas to loosely structured markdown. In my experience with LLMs for product managers, markdown becomes a flexible intermediate representation that’s easy to diff, trace, and feed back into models without brittle parsing.

Equally important, they use customer feedback—from calendar approvals to regeneration requests—as their primary evaluation signal. That’s eval-driven development in practice: close the loop with lightweight evals that reflect genuine user intent, not proxy metrics.

The planning model is elegant: the three mini-calendars—public events, business-specific events, and recommended campaigns—roll up into a coherent plan that eliminates the blank-page problem and enables steady, predictable execution.

Crucially, they’re building traceability so customers can see which context documents influenced their content. This kind of transparency increases trust, accelerates edits, and supports governance in regulated categories where auditability matters.

Onboarding and data collection stay pragmatic: let the system crawl first, ask humans only for deltas, and progressively profile over time. It’s a pattern I advocate in continuous discovery and AI workflows—keep humans in the loop without overwhelming them, and make the right action the easy action.

Early on, they used Simon Sinek's Golden Circle framework to validate demand and sharpen messaging. Framing the "why" before the "what" helps teams maintain a crisp value proposition and tighten their go-to-market strategy.

Performance measurement goes beyond vanity metrics by connecting marketing performance back to point-of-sale data for attribution. The ability to tie campaigns to revenue events is the bridge from clever content to accountable outcomes.

What’s next is equally compelling: deeper attribution, omnichannel expansion, and digital out-of-home displays. For SMBs, that points to a unified analytics platform spanning email, social, and in-store touchpoints—exactly where modern marketing is headed.

My takeaways for builders: invest in a retrieval-first pipeline with a resilient document hierarchy; prefer loosely structured markdown over rigid JSON when dealing with messy inputs; design human-in-the-loop controls that double as evals; and always connect activity to business outcomes. That’s how you turn an idea into a repeatable system that scales.

If you want to explore further, start here: Mowie AI — AI marketing platform for SMBs. For early validation and storytelling, revisit Simon Sinek's Golden Circle.

Inspired by this post on Product Talk.

December 11, 2025
Automated Insights for Product Teams: Uncover Causal ‘Aha’ Moments in Minutes, Not Weeks

I’ve spent countless cycles guiding teams through the maze of dashboards, SQL pulls, and ad‑hoc analyses—only to watch truly meaningful patterns emerge far too late. Automated insights are the next frontier in product analytics: a shift from manual exploration to AI that proactively surfaces what matters most. When we let the system do the heavy lifting, we accelerate discovery, reduce bias, and give product trios the clarity to act.

Finding causal connections in product data involves exhaustive searches and tests. We trained our AI to find “aha” moments in minutes instead of weeks.

Here’s what that means in practice for product management: the platform continuously scans events, cohorts, and segments; prioritizes signals linked to activation, conversion, and retention; and highlights likely causes behind meaningful movements in your core KPIs. Instead of sifting through endless funnels and cohorts, I get ranked hypotheses I can validate with targeted A/B testing and minimum detectable effect (MDE) guardrails.

This approach turns analytics into action. Automated insights reduce time-to-learning, tighten our discovery loops, and make continuous discovery tangible—especially when we’re aligning roadmaps, designing experiments, and refining onboarding. Whether you’re using tools like Amplitude analytics or instrumenting a unified analytics platform, the value is the same: faster, clearer paths to customer impact.

I’ve seen teams unlock retention analysis breakthroughs by spotting counterintuitive patterns—like a specific feature combination or an overlooked step in onboarding—well before they would have surfaced through manual analysis. With AI workflows scanning the noise and elevating the signal, we can focus on decisions: ship or iterate, scale or sunset, double down or pivot. That’s empowered product teams in action.

If you’re building for product-led growth, this is the leverage you’ve been waiting for. Automated insights transform how we prioritize, test, and communicate strategy—bringing us from gut feel and lagging indicators to explainable, causal narratives we can stand behind. The outcome is simple: more confident bets, less waste, and a faster path to durable product-market fit.

Inspired by this post on Amplitude – Best Practices.

December 10, 2025
Unlock Real-Time Product Insights: Amplitude + OpenAI MCP in ChatGPT, Without BI Bottlenecks

I’ve been working to remove the friction between product questions and product answers. The most impactful step so far: connecting Amplitude analytics directly into ChatGPT via OpenAI’s MCP. This turns everyday conversations into decision-grade insights—no dashboards to hunt, no SQL to write, and no analytics queue to wait on.

Connect Amplitude data directly to the tools your team uses every day. OpenAI’s MCP connector eliminates traditional barriers to product data.

In practice, this means I can ask ChatGPT natural-language questions like, “Where are users dropping in our activation funnel this week?” or “Which cohorts are driving retention lift post-onboarding?” and get grounded answers from Amplitude—fast. It’s a step-change for product-led growth because the insights live where we already think and plan.

Here’s how I apply it day to day: I’ll prompt ChatGPT to compare week-over-week activation for new SMB signups across regions, diagnose drop-offs by step, and summarize A/B testing outcomes with guardrails like minimum detectable effect considerations. When we’re shaping strategy, I’ll pull a retention analysis and cohort breakdown to inform bet sizing and roadmap tradeoffs—all without pulling the team into a BI bottleneck.

Governance remains non-negotiable. I scope the MCP tools to a least-privilege data slice, apply privacy-by-design rules to exclude PII, and log every query for auditability. Clear data governance and AI risk management policies ensure we maintain trust while accelerating discovery. Tight context window management keeps prompts focused and reduces noise.

Operationally, the setup is straightforward: define the MCP tool spec for Amplitude, map canonical events and metrics (activation, retention, conversion, and product-qualified lead stages), and test with a retrieval-first pipeline so responses reliably cite the right source of truth. We standardize metric definitions across product, growth, and customer success to avoid semantic drift.

The impact on empowered product teams is immediate. Continuous discovery becomes a daily habit rather than a quarterly ritual; questions move from “I’ll get back to you” to “Let’s check right now.” For product managers working with LLMs, this is the connective tissue that makes ChatGPT a true ChatGPT connector for analytics—an on-demand, unified analytics platform that supports faster iteration and sharper decision-making.

If you’ve been waiting to make analytics truly ambient, this is the moment. Start small with a single funnel or cohort, validate governance, and expand to your core lifecycle metrics. The payoff is a shared understanding of what’s working, what’s not, and where to focus next—delivered in the flow of work.

Inspired by this post on Amplitude – Best Practices.

December 10, 2025

Operationalizing AI: A Practical System for Scalable Growth

Your AI pilot works in the demo. Then it reaches a live workflow and slows down: the data is incomplete, nobody owns the exceptions, reviewers apply different standards, and the team cannot prove whether the result improved revenue, cost, speed, or retention.

The gap is not model quality alone. Scalable growth requires an operating system around the model: a constrained business outcome, a mapped workflow, approved data, explicit decision rights, measurable quality, controlled releases, and a path for handling failure. Build those pieces around one valuable use case, and AI can become a repeatable business capability instead of a collection of pilots.

Choose the growth constraint before the AI use case

Do not begin with a broad instruction to “find an AI use case.” That framing encourages teams to start with a model capability and search for somewhere to place it. Start with a constrained business problem instead.

The unit of investment should be a decision or task inside a customer or employee journey. “Build a churn copilot” is too broad. “Before a renewal review, summarize approved usage and CRM signals, identify the evidence of risk, and propose an action for the customer success manager to review” is narrow enough to test.

Most growth-oriented opportunities fit into four useful lanes:

Revenue: improve qualification, conversion, expansion, cross-sell, or win-back decisions. Measure the commercial event, not the number of AI recommendations generated.
Efficiency: reduce the cost, handling time, rework, or backlog associated with a repetitive process. Good candidates have high task volume and outputs that can be checked without recreating the work.
Speed: shorten a discovery, delivery, or release cycle. If the workflow serves software delivery, deployment frequency can be relevant, but it is not evidence of customer or commercial value by itself.
Activation and retention: make onboarding, guidance, or support more contextual. Measure whether customers reach the intended product behavior and continue receiving value, not whether they clicked an AI-generated tooltip.

A disciplined portfolio can pair one revenue use case with one efficiency use case, define success before development, and release each through a narrow MVP. That balance matters. An efficiency-only roadmap can shrink costs without creating differentiation, while an unconstrained revenue bet can consume attention without proving economic value.

Screen each candidate with the same questions:

What business metric should move, and what is its current baseline?
Which person, decision, and moment in the workflow create that movement?
Does the task occur often enough to justify a reusable solution?
Are the required inputs available, current, and approved for this purpose?
Can a reviewer distinguish an acceptable result from an unacceptable one?
What happens when the system is wrong, and can the action be reversed?
Who owns the outcome after the launch team moves on?

My test is blunt: if you cannot name the workflow event, the owner, the baseline, and the failure consequence, you do not yet have an implementation candidate. You have a discovery question. Fund the learning needed to answer it before funding scale.

Convert the use case into a controlled workflow

An AI feature becomes operational when its behavior is defined inside the surrounding work. That means understanding what happens before the model is called, what the model may do, how its output is checked, and what happens next.

Begin by mapping the task as it is performed, choosing one step to augment, selecting the right automation method, and iterating against an explicit quality bar. Do the task manually while mapping it if the real process is unclear. Policy documents often describe the intended path; observation reveals the exceptions that determine whether automation will survive production.

Name the trigger. Specify the event that starts the workflow, such as a support request, renewal review, onboarding milestone, invoice submission, or product release.
Identify the inputs. Record each system, document, field, permission, and freshness requirement. Separate required evidence from optional context.
Expose the decisions. Write down the classifications, judgments, calculations, and approvals a person currently makes. Hidden judgment is where apparently simple automations tend to break.
Specify the output. Define its schema, audience, channel, timing, and acceptable evidence. “Produce a helpful answer” is not a specification.
Map exceptions. Include missing records, contradictory inputs, unsupported requests, low-confidence cases, policy conflicts, and unavailable downstream systems.
Assign each step to code, retrieval, an LLM, or a person. The workflow should use the simplest reliable mechanism for each job.
Define the handoff. State who reviews the result, what they can change, when the workflow must stop, and where failures are recorded.

Use each form of automation for the work it can control

Use deterministic code for exact calculations, validation rules, permissions, routing, and other behavior that should produce the same answer from the same inputs. Use an LLM where language is ambiguous, inputs are unstructured, or the task requires drafting, summarizing, extracting, or classifying meaning.

When the answer must reflect company facts, policy, or customer history, retrieve the approved information at runtime instead of expecting the model to remember it. A retrieval-first design can connect behavioral and CRM context to account signals and recommended actions, while preserving a visible trail back to the evidence used.

Keep a person in the path when the consequence is material, the action is difficult to reverse, or the definition of a correct result remains contested. Human review is not a permanent excuse for weak quality, however. The reviewer needs defined criteria, enough context to make a decision, and an easy way to correct and categorize the failure.

Write an execution contract, not just a prompt

A production instruction set should define more than tone and role. Treat it as an execution contract containing:

the objective and the business context;
the permitted inputs and authoritative evidence;
the decision criteria the system must apply;
the required output structure;
the actions it may and may not take;
the conditions that require refusal or escalation;
the way uncertainty should be represented;
examples of acceptable, unacceptable, and edge-case behavior.

For an agentic workflow, increase authority in deliberate stages: observe, draft, recommend, act after approval, and only then act within defined limits. Do not jump from a convincing chat demonstration to autonomous execution. Agentic AI needs explicit guardrails and verifiable quality before it can safely take work out of a human queue.

Measure business value, workflow performance, and AI quality separately

A dashboard that reports requests, tokens, or generated answers tells you that the feature was used. It does not tell you whether the business improved. You need separate measures because an AI system can look healthy at one layer while failing at another.

Measurement layer	What to track	What it reveals
Business outcome	Conversion, expansion, cost per completed outcome, cycle time, activation, or retention	Whether the investment affects the growth constraint it was chosen to address
Workflow performance	Completion, rework, exception, escalation, abandonment, and end-to-end latency	Whether the surrounding process can absorb and use the AI output
AI quality	Correctness, evidence support, instruction adherence, output validity, and appropriate refusal	Whether the system behaves acceptably across expected and difficult cases
Risk and operations	Unauthorized data exposure, prohibited actions, overrides, incidents, rollback events, and unresolved failures	Whether growth is being purchased with unacceptable operational or trust costs

Build the measurement path before the rollout:

Capture the baseline. Measure the existing workflow using the same outcome definition you will use after launch. Otherwise, a faster AI step can hide slower review, higher rework, or shifted labor elsewhere.
Create a representative evaluation set. Use permitted examples from normal, difficult, and failure-prone cases. Define the expected result and the critical errors for each case.
Weight failures by consequence. Formatting errors, unsupported factual claims, privacy failures, and unauthorized actions should not disappear into one average score.
Run offline evaluations before exposure. Test the complete combination of instructions, model, retrieval, tools, and output validation. A model score alone does not represent the production system.
Release behind a feature flag. Start with a controlled cohort, preserve the ability to roll back, and compare outcomes. Use A/B testing when assignment and outcome measurement are credible; use a phased rollout when they are not.
Record versions. Log the model, instructions, retrieval configuration, tools, and policy version associated with each result so a regression can be traced.
Turn failures into future tests. Categorize meaningful production failures and add them to the evaluation set before the next release.

This is the practical meaning of eval-driven development: instrument the system, watch for drift, and tighten the delivery loop while changes remain controlled by feature flags. It turns evaluation from a launch checkpoint into part of product development.

Use a scale gate that includes economics

Do not scale because the demo is impressive or employees like the interface. Require four decisions:

The business outcome is moving in the intended direction, or there is credible evidence that the workflow is producing the leading behavior tied to it.
Quality remains acceptable across normal cases, edge cases, and high-consequence failures.
Total cost per successful outcome is viable after model usage, retrieval, storage, human review, escalation, rework, and operations are included.
The operating owner can detect, contain, and learn from failures without depending on the original project team.

If a pilot fails one of these gates, the decision is not automatically to cancel it. Narrow the scope, change the workflow, improve the evidence, or stop. What matters is that expansion is earned by measured behavior rather than assumed from adoption.

Scale through guardrails, reusable components, and clear ownership

Governance should make routine decisions faster. When every team has to rediscover which data is permitted, which evaluation is sufficient, and who can approve a release, governance becomes a sequence of meetings. When those expectations are encoded in a standard launch record, teams know the path before they build.

Create a minimum launch record for every workflow

the business outcome, baseline, and accountable owner;
the workflow boundary, users, and authorized actions;
the approved data sources, access controls, retention rules, and prohibited data;
the evaluation set, acceptance criteria, and critical failure classes;
the human review and escalation conditions;
the logging, monitoring, feature flag, and rollback plan;
the model, retrieval, tool, and vendor dependencies;
the incident owner and the method for notifying affected internal teams or customers when appropriate.

Privacy-by-design, data governance, red-teaming, and defined review gates are growth infrastructure. They reduce repeated risk debates and make the safe path reusable across launches.

If a workflow touches personal data, confidential customer content, employment decisions, payments, security actions, or contractual commitments, involve the appropriate privacy, security, legal, financial, or people owner before live use. The downside is not limited to a poor answer. The workflow can expose restricted data or take an action the business cannot easily reverse.

Assign ownership beyond launch

Four responsibilities must be explicit, even when one person holds more than one:

Business outcome ownership: decides whether the workflow is worth continuing based on the target metric and economics.
Workflow ownership: manages exceptions, reviewer behavior, process changes, and user feedback.
Technical ownership: controls releases, versions, integrations, reliability, monitoring, and rollback.
Risk ownership: defines the policy boundary and approves material changes to data, authority, or exposure.

This prevents a common operating failure: the product team treats launch as completion, while the operations team inherits a changing probabilistic system without the tools or authority to manage it.

Standardize the recurring parts, not every local process

Once working use cases expose recurring needs, turn those needs into shared capabilities. Useful candidates include identity and permissions, governed retrieval connectors, evaluation tooling, instruction and model versioning, observability, feature flags, rollback controls, and cost attribution.

Keep the final workflow close to the business team that understands the customer, exceptions, and outcome. Centralize the controls and infrastructure that should be consistent. This creates leverage without forcing every function into the same process.

Review the portfolio as a set of products, not permanent projects. The decision for each workflow should be to expand it, fix a known constraint, narrow its authority, or retire it. Continuous discovery with product trios can refine the prompts, data sources, and experience while evidence determines what scales and what stops.

Operationalizing AI: three questions leaders ask

Should you build a central AI platform first?

Usually, no. Start with the minimum secure infrastructure required for a valuable workflow. Standardize a component when several use cases need the same capability or when inconsistency creates material risk. Data access, identity, logging, and release controls may need early consistency; a broad internal platform without proven workflows can become an expensive set of assumptions.

How do you know a pilot is ready to scale?

A pilot is ready when it improves the intended business or workflow outcome, stays within quality and risk boundaries, has viable cost per successful outcome, and can be operated without daily intervention from its builders. Usage and positive comments are supporting signals, not a scale decision.

Where should a human remain in the loop?

Keep human approval where consequences are high, actions are difficult to reverse, evidence is incomplete, or acceptable judgment cannot yet be specified. Remove or reduce review only when evaluations and production monitoring show that the remaining risk is understood and controlled. A reviewer who merely clicks approve without adding judgment is not a guardrail; it is latency disguised as governance.

For your next AI proposal, require a one-page charter containing the outcome, workflow boundary, owner, baseline, approved data, evaluation set, failure policy, release plan, and full cost model. If a line is blank, fund discovery to resolve it. If the charter is complete, release the smallest useful workflow behind a control, learn from real failures, and widen its authority only when the evidence earns it.

References

December 10, 2025

Beyond Accuracy: The Trust-First Evaluation Metrics I Use to Scale High-Impact AI Products

When I assess whether an AI product is ready for prime time, I start with trust—not model accuracy. Accuracy is table stakes; trust is what earns adoption, drives retention, and unlocks durable product-led growth.

Evaluation metrics in AI products go beyond accuracy. Learn how product teams use trust-driven metrics to build reliable, growth-driving AI systems.

In practice, I organize trust-driven metrics into four layers: model quality and safety, user and business outcomes, operational reliability and cost, and governance and compliance. This layered approach keeps product trios aligned on what matters now, what must be gated in CI/CD, and what signals we’ll use to prove progress against outcomes vs output OKRs.

On model quality and safety, I care about precision, recall, F1, calibration, and abstention behavior, but also the hard-to-fake signals: hallucination rate, grounding and faithfulness, citation coverage, toxicity, bias, and fairness. For generative systems, I instrument refusal correctness (declining unsafe requests) and evidence adequacy (did the answer rely on retrieved, trustworthy sources).

User and business outcomes must be explicit. I track adoption, activation, task success rate, time to first value, win rate uplift in assisted workflows, CSAT and NPS deltas, and retention analysis by cohort exposed to AI features. For customer support scenarios, deflection rate, average handle time change, and first-contact resolution are core; for sales or ops copilots, I monitor cycle-time reduction and error-rate reduction in critical tasks.

Experimentation is non-negotiable. I design A/B testing with a clear minimum detectable effect (MDE), pre-registered guardrails for safety and quality, and sequential tests that stop early if harm outpaces benefit. Online metrics are always paired with offline evals so we can iterate quickly without exposing users to regressions.

Operationally, trust shows up as speed, stability, and cost predictability. I track latency end-to-end, time to first token, throughput, rate of 5xx and timeouts, cost per request, and caching effectiveness. We also trend safety incidents per 10,000 interactions and mean time to mitigation to keep reliability visible alongside performance.

Governance and compliance are part of the product, not an afterthought. Data governance and privacy-by-design metrics include PII exposure rate, data lineage coverage, access-control correctness, audit pass rate against internal policies, and model and prompt change traceability. This is the backbone of our AI risk management posture and accelerates regulatory compliance reviews instead of slowing them down.

The delivery engine for all of this is eval-driven development. We maintain golden datasets and scenario-based test suites that mirror real user intents, gate releases in CI/CD with minimum thresholds, and run canary rollouts to validate offline–online alignment. Every model or prompt update gets a comparable scorecard so product, engineering, and design can trade off quality, speed, and cost with shared facts.

For LLM-heavy features, retrieval-first pipeline metrics are mandatory. I monitor retrieval hit rate, recall at K, mean reciprocal rank, context contamination, and citation correctness. With large prompts, context window management matters: we track context utilization, truncation rate, and the contribution of each context block to final answers to avoid silently losing critical evidence.

Finally, trust must be legible. I package these metrics into an executive scorecard that maps to business outcomes, risk appetite, and OKRs, with clear thresholds for ship, improve, or roll back. When teams can articulate trade-offs—say, a 20% latency reduction at a small cost increase, or a lower hallucination rate at the expense of higher abstention—they build credibility with stakeholders and confidence with customers.

Trust is not a single number; it’s a system of evidence. By instrumenting these layers and operationalizing AI Strategy with rigorous, transparent metrics, we can ship faster, reduce surprises, and earn the right to scale AI features across the product portfolio.

Inspired by this post on Product School.

December 8, 2025
From No-Code Hack to 10,000 Weekly Calls: Inside Perk’s Voice AI That Actually Works

I love real-world AI that ships, scales, and actually solves painful customer problems. This story checks every box. As a product leader who has brought agentic AI to production environments, I was captivated by how a small, focused team at Perk took a no-code voice AI prototype and turned it into a system that reliably makes 10,000+ calls per week to prevent failed hotel payments.

What happens when you combine a real customer problem, a no-code prototype, and a team willing to listen to every single call?

Steven Payne (Product Manager), Gabriel Stock (Senior Engineering Manager), and Philipe Steiff (Senior Software Engineer) from Perk share how they built a voice AI agent that calls hotels to verify virtual credit card payments, preventing travelers from arriving to find their rooms unpaid. This is a textbook example of linking operational pain to a high-leverage AI solution.

What started as a hackathon experiment in Make.com became a production system handling over 10,000 calls per week across multiple languages. Along the way, the team learned hard lessons about prompt engineering for voice (numbers, pronunciation, and a very "Karen-like" first version), how to break a single monolithic prompt into structured conversation stages, and why listening to actual calls beats any amount of theorizing.

From a product management perspective, this approach aligns perfectly with eval-driven development and continuous discovery. Structure the problem, instrument aggressively, ship safely, then listen—deeply—to real interactions. In my own teams, I’ve seen that nothing accelerates iteration on agentic AI like closing the loop between qualitative call reviews and quantitative evals.

They built a working prototype without writing a single line of backend code.

They structured the call into discrete stages (IVR, booking confirmation, payment) to improve reliability.

They created two eval systems: one for call success classification, another for conversational behavior.

They scaled from five calls a day to tens of thousands per week while maintaining quality.

This is a detailed look at building AI for real-time human interaction—where the stakes are high and the feedback is immediate.

Guests: Steven Payne, Product Manager, Perk; Gabriel Stock, Senior Engineering Manager, Perk; Philipe Steiff, Senior Software Engineer, Perk.

What stood out to me was how Perk's team identified an AI use case by connecting prior experimentation with a real operational problem. Why they chose Make.com for prototyping—and shipped to production without touching backend code—underscores how far no-code can take you when paired with crisp problem framing. The evolution from a single prompt to structured conversation stages (IVR handling, booking confirmation, payment request) is exactly how you harden agent behavior for production.

Breaking up the agent's task dramatically improved reliability. They also built two eval systems: classification for success rates and LLM-as-judge for conversational behavior. Even with automation, the team still listens to calls manually—a practice I strongly endorse for uncovering edge cases, trust issues, and UX nuances that dashboards can’t show.

The challenge of prompt engineering for voice—numbers, booking references, and text-to-speech markup—was non-trivial. Expanding to German revealed that prompts in native language improve results. And, as often happens with operations-heavy rollouts, this project uncovered other operational problems they didn't know existed—valuable signal for the roadmap.

Resources & Links: Perk. Make.com — No-code automation platform used for the prototype. Twilio — Voice/telephony provider. Eleven Labs — Text-to-speech provider (used in early experiments).

Chapters: 00:00 Introduction to the Team; 01:54 Understanding PERK's Mission; 02:59 Challenges in Travel Booking; 07:27 AI Solutions for Customer Care; 09:52 Prototyping with AI and Voice; 17:00 Implementing AI in Production; 25:51 Learning Through Trial and Error; 26:40 Prompting Challenges and Solutions; 27:58 Iterating on Prompts and Evaluations; 30:08 Scaling and Production Challenges; 32:43 Advanced Evaluation Techniques; 35:32 Real-World Applications and Success; 49:07 Future Directions and Expansion; 53:53 Conclusion and Team Reflections.

My product takeaways: Start with clear operational pain and measurable outcomes (e.g., payment verification). Use no-code to validate quickly, then progressively harden. Treat voice AI like any production system: break it into deterministic stages, add guardrails, and measure both outcome and behavior. Pair automated evals with hands-on reviews. And when going multilingual, write prompts in the native language—your accuracy will thank you.

If you’re exploring agentic AI for operations, this is the blueprint: tight scoping, Make.com for speed, Twilio for reliability, structured prompts for control, and an eval-driven loop to scale quality with confidence.

Inspired by this post on Product Talk.

December 4, 2025

How Startups Earn Visibility in ChatGPT and Perplexity

A prospect asks ChatGPT or Perplexity for the kind of product you sell. Several competitors appear. Your startup does not. That does not automatically mean your product is weak or your SEO has failed. It often means the system cannot find enough clear, consistent, and corroborated evidence to include you confidently.

Your job is not to force your company into every answer. It is to make your startup easy to identify, accurately categorize, and safely recommend when it genuinely fits the question. That requires coordinated work across positioning, content, technical structure, third-party proof, and measurement.

Key takeaways

Measure visibility across important buyer questions, not as one universal AI-search ranking.
Build a page for each major decision: category, use case, integration, price, comparison, and deployment risk.
Make important claims explicit in visible HTML, then reinforce them with accurate metadata and schema.
Support first-party claims with reviews, partner pages, case studies, documentation, and other independent evidence.
Use a stable prompt set to find specific visibility failures, change the relevant evidence, and retest.

Measure recommendation coverage, not an imaginary rank

Conventional search encourages a positional question: where do I rank? AI search requires a different question: for which buyer decisions can the system understand and support a recommendation of my product?

AI search behaves more like a synthesis engine than a page of ranked blue links. It assembles an answer around the wording and context of a prompt. Change the question from best software for a category to best software for a particular team, workflow, integration, budget, or risk profile, and the eligible recommendations may change.

There is therefore no single visibility score that tells the whole story. A startup can be visible for category discovery but absent from integration questions. It can be named as an alternative yet omitted when the buyer adds a security requirement. It can also be mentioned with an outdated description, which is exposure without useful discovery.

A practical baseline should distinguish four outcomes:

Discovery: Does your company appear when the prompt describes a problem you solve?
Positioning: Is it placed in the right category and associated with the right audience and job?
Fit: Does the answer explain when your product is appropriate, including relevant trade-offs?
Evidence: Are the supporting claims current, specific, and connected to credible pages?

Start with the questions that already matter in your buying journey. Include category exploration, problem framing, use-case fit, integrations, commercial value, alternatives, and deployment risk. Preserve the exact wording of each prompt. If you rewrite the test every time, you will not know whether your evidence improved or the question merely changed.

Record more than whether your name appeared. Save the product description, recommendation context, claims, citations, omissions, and factual errors. A mention is not a win if the answer sends the wrong buyer to your product or attributes a capability you do not offer.

Turn buyer intent into an answerable page system

Many startups try to solve AI visibility by publishing more blog posts. Volume is rarely the first constraint. The more common problem is that the website has no precise page capable of answering the buyer’s actual question.

Your homepage cannot carry the entire decision journey. Give each high-value intent a clear destination:

Buyer decision	Question the page must answer	Best page type	Evidence to include
Category exploration	What is this product, and who is it for?	About or category page	Plain category definition, target customer, core job, and differentiator
Problem framing	How should I understand and solve this problem?	In-depth explainer	Method, terminology, constraints, and links to primary material
Solution fit	Can this product handle my workflow?	Use-case page	User, workflow, inputs, outputs, limitations, and customer evidence
Integration fit	Does it work with the rest of my stack?	Integration page or documentation	Prerequisites, supported connection, setup steps, data flow, and known limits
Commercial fit	What will I pay, and what value should I expect?	Pricing and value page	Pricing structure, inclusions, exclusions, assumptions, and verifiable outcomes
Competitive choice	When should I choose this product instead of an alternative?	Comparison or alternatives page	Points of parity, meaningful differences, trade-offs, and cited claims
Deployment risk	Can my organization use it safely?	Trust center	Security, privacy, compliance, governance, and data-handling information

Each page should lead with a direct answer. Do not make a retrieval system infer your category from a slogan or reconstruct an integration from a press release. A useful positioning sentence follows a simple structure: [Product] is a [category] for [audience] that needs to [job], distinguished by [relevant difference]. Use the same underlying definition wherever the product is introduced.

Use-case pages need more than a collection of benefits. Name the user, triggering problem, workflow, expected output, dependencies, and boundaries. If the product is suitable only under particular conditions, state them. Precise qualification can reduce superficial visibility while improving the quality of the recommendations that remain.

Integration pages deserve the same care. A logo wall proves very little. Explain what connects, in which direction data moves, what setup requires, and which workflows the connection supports. Link to technical documentation and the partner’s corresponding page when one exists.

Comparison pages should help a buyer make a decision, not manufacture a victory. Start with the shared category, acknowledge points of parity, identify the conditions that make each option a better fit, and cite claims that a reader can verify. A fair statement such as one product suits a particular workflow while another suits a different operating model is more useful than an unsupported declaration that yours is best.

Transparent pricing matters for the same reason. If a public amount is not available, you can still explain the pricing unit, packaging logic, included capabilities, major variables, and purchasing path. The aim is to remove avoidable ambiguity from a commercial-fit question.

Make the corpus easy to retrieve and hard to misread

Good information can remain invisible when it is buried in a PDF, hidden behind vague navigation, contradicted by metadata, or scattered across pages with no canonical version. Retrieval-friendly content reduces the work required to locate, segment, and interpret an answer.

Work through the site in this order:

Make the visible narrative consistent. Use the same product name, category, audience, and core capability across the homepage, About page, product pages, documentation, and trust center. Resolve genuine contradictions before adding markup.
Give every important answer a stable URL. Use descriptive headings, short focused sections, sensible internal links, and linkable anchors. Keep documentation in HTML when possible, even if you also offer a PDF.
Add schema that describes the visible page. Organization, Product, FAQPage, HowTo, and Article JSON-LD can clarify entities and content types when they accurately match what a person can read on the page.
Align the surrounding signals. Titles, meta descriptions, canonical URLs, and Open Graph data should reinforce the same identity and purpose rather than introducing alternate names or claims.
Remove retrieval friction. Maintain a clean sitemap, review robots.txt for accidental blocking, keep important pages reachable through navigation, and provide fast mobile-first experiences.
Keep technical material usable. Provide copyable commands, configuration examples, prerequisites, expected results, and failure conditions where they are relevant.

Schema is a translation layer, not evidence. Product markup cannot rescue an unsupported claim, and FAQ markup cannot turn a thin sales page into an authoritative answer. Add structured data after the visible content is accurate and complete.

The trust center is especially important for B2B products. Security, compliance, privacy, governance, and data-handling questions often enter the buying process before a prospect speaks to sales. Give each topic a clear, current answer. Avoid mixing aspirational commitments with controls that are already in place.

Freshness also needs visible ownership. Release notes should reflect material product and integration changes. Outdated feature claims should be corrected or retired instead of left to compete with the current version. Schedule a quarterly review of commercially important pages, documentation, comparison claims, and trust material. The goal is not to alter dates cosmetically; it is to ensure that the underlying answer remains true.

Earn corroboration where your company cannot control the wording

Your website establishes what you claim. Independent surfaces help establish whether anyone else has reason to believe it. That distinction becomes important when a recommendation involves operational risk, meaningful spend, or a crowded category.

Map each commercially important claim to the strongest available proof:

Adoption: detailed customer stories, current review profiles, and customer outcomes with verifiable metrics.
Compatibility: partner directories, joint integration pages, and documentation that confirms the supported connection.
Technical maturity: accessible documentation, maintained repositories where relevant, and README files that accurately explain installation and use.
Category authority: reputable industry mentions, analyst coverage, or citations by practitioners and institutions with relevant expertise.
Deployability: security, compliance, governance, and privacy material that a buyer can inspect rather than a generic statement that the product is secure.

Do not chase mentions indiscriminately. A third-party page is useful when it verifies a claim a buyer cares about. An integration listing that confirms compatibility can be more valuable for an integration prompt than broad publicity that says nothing about the product’s operation.

Case studies should make their evidentiary limits visible. Identify the customer context, starting problem, product use, measured result, and method behind the metric. If the outcome is self-reported or cannot be independently verified, describe it that way. Specificity makes the claim easier to evaluate; inflated certainty makes the entire corpus less trustworthy.

Build a proof inventory before launching another content campaign. For each positioning claim, record the first-party explanation, customer evidence, independent corroboration, current URL, owner, and freshness status. Empty cells reveal whether you have a writing problem, a product-evidence problem, or a distribution problem.

This inventory also prevents a common sequencing mistake. A startup may publish many pages around a claim that no customer, partner, reviewer, or technical artifact supports. More repetition does not create stronger evidence. First establish the truth of the claim, then make that truth easy to discover in the places a recommendation system can retrieve.

Run AI visibility as an eval-driven product loop

AI-search work becomes vague when the team alternates between random prompts and random content changes. Treat the discovery experience as a product surface with defined test cases, observable failures, and controlled iterations.

Define a stable prompt set. Represent the buyer intents you want to serve, using the language a real evaluator would use at each decision stage.
Capture a baseline in ChatGPT and Perplexity. Record the exact prompt, system, test date, answer, recommendation context, cited pages, and factual errors.
Classify the failure. Distinguish absence from miscategorization, weak fit evidence, missing corroboration, stale information, or retrieval of the wrong page.
Change the evidence connected to that failure. Improve the category definition for a positioning error, an integration page for a compatibility gap, or the trust center for an unsupported deployment answer.
Rerun the same test cases. Look for improved coverage and accuracy without assuming that a single response proves a durable change.
Connect visibility to buyer behavior. Track referrals from AI-driven surfaces, landing-page engagement, qualified demand, and pipeline where your analytics can identify them responsibly.

Use a simple evaluation record rather than one blended score. Mark whether the product was present or absent, whether its category was correct or wrong, whether fit was supported or merely asserted, whether citations were current, and whether the linked page offered a useful next step. Separate fields tell you what to fix. A single number hides the cause.

Answer variability is part of the environment, so treat one run as an observation rather than a verdict. The useful signal is whether the same class of important prompts becomes more consistently accurate after you improve the relevant material.

A/B testing can help when a page receives enough appropriate traffic and the change can be measured through user behavior. Test answer placement, headings, proof presentation, or the route to a next step. Do not A/B test incompatible facts about what the product is. Positioning consistency is a prerequisite for the evaluation, not an experiment variant.

Avoid the shortcuts that create activity without evidence: bulk publishing shallow pages, applying every available schema type, writing hostile comparison copy, leaving essential documentation only in PDFs, and reporting raw mentions without checking accuracy or commercial relevance.

In your next working session, choose the buyer question closest to an active product decision. Inspect the answer, identify the missing or unreliable evidence, improve the page that should resolve it, and add one credible corroborating signal. Then preserve the prompt and retest it. Repeating that loop across the decision journey is how AI visibility becomes an operating capability instead of a one-time content project.

References

Amplitude – Crack the AI Search Code: How Startups Win Recommendations in ChatGPT and Perplexity

December 3, 2025

How Amplitude AI Feedback Turns Noise into Product Signal You Can Ship With Confidence

I’ve spent enough time in the trenches of product management to know the hardest part isn’t collecting feedback—it’s separating signal from noise. When every channel is buzzing, the real question becomes: what should we build next, and why? That’s where Amplitude AI Feedback has changed how I work. It gives me a disciplined, data-informed way to turn messy qualitative input into clear, defensible roadmap decisions.

Learn how Amplitude AI Feedback leverages AI to transform massive volumes of customer feedback into actionable product insights.

In practice, this means I can synthesize input from support tickets, NPS responses, user interviews, sales notes, and reviews—then connect those insights to product behavior data from Amplitude analytics. The result isn’t just a list of requests; it’s a ranked problem set grounded in evidence, which makes product discovery and continuous discovery faster, clearer, and less biased.

A recent example: we were hearing recurring complaints about onboarding friction, but it wasn’t obvious which steps truly mattered. By pairing feedback themes with activation and retention signals, I could zero in on the first-session setup tasks that correlated with drop-off. That clarity guided product roadmapping and sprint planning decisions we could stand behind, and it accelerated user activation without bloating the backlog.

My workflow is straightforward: aggregate feedback, cluster themes, validate with behavioral metrics, and translate insight into outcomes. I look for patterns tied to user activation, retention analysis, and moments that drive product-led growth. When the evidence shows a request is both frequent and high-impact, it earns a place on the roadmap; when it’s loud but low-impact, it becomes a targeted experiment rather than a default commitment.

What I appreciate most is the confidence this brings to stakeholder conversations. Instead of debating opinions, we review the evidence: quantified themes, clear user stories, and measurable KPIs. That turns “Finally, Signal That Tells You What to Build” from a slogan into an operating principle, and it helps empowered product teams move faster with fewer reversals.

If you’re building your AI Strategy or exploring LLMs for product managers, this is one of the highest-leverage moves you can make: use a unified analytics platform to connect qualitative feedback with quantitative behavior. It sharpens prioritization, improves time-to-learning, and keeps the team focused on outcomes—not outputs.

Inspired by this post on Amplitude – Best Practices.

December 2, 2025
Own Your AI: 4 Essential Roles to Supercharge Support and Prevent Performance Drift by 2026

AI doesn’t fail because the model is bad, it fails because ownership is missing.

When someone truly owns your AI, everything changes. Resolution and automation rates climb, the system self-improves, and the customer experience transforms in ways a dashboard alone will never show you.

This is part three of our five-part series on customer service planning for 2026. We’ll be sharing all five editions on our blog and on LinkedIn.

If you’d rather have them emailed to you directly as they’re published, drop your details here.

Last week, we introduced the four roles that make AI actually work in a support organization. These roles are already showing up inside the teams who are scaling AI the fastest, and this week, we get closer to the ground.

Here’s what these roles look like in practice — what they do, how they work, and why your AI performance will inevitably drift without them.

AI operations lead — owns AI performance, every day. I think of this person as the air-traffic controller for our AI Agent. I treat the AI as a living system that needs ongoing supervision, evaluation, and tuning. This role is accountable for what leaders care about most: quality, reliability, and continuous improvement.

The AI ops lead sees the whole picture: conversation quality, missing knowledge, flawed assumptions, unexpected failures, new opportunities for automation, and the subtle signals that the system is beginning to drift. In practice, that vigilance is the difference between steady gains and slow decline.

Day-to-day, here’s what I expect from this role.

1. Reviews AI conversations and surfaces performance patterns. The AI ops lead monitors the AI Agent’s behavior — the tone shift after a product launch, a sudden dip in resolution for a specific intent, or conversation clusters revealing new customer behavior. They scan for anomalies, trends, and early warnings, with an emphasis on what’s happening right now, not last week. Without this intentional ownership, I’ve watched a 2% dip turn into a 10% drop in days.

2. Prioritizes fixes and improvements. Once patterns emerge, they triage fixes like a product team handles bugs. Missing or incorrect content? They route it to the knowledge manager. Behavioral issues? They adjust guidance and guardrails. Action or system issues? They partner with the automation specialist. This connective tissue turns individual fixes into compounding improvements.

3. Defines and maintains AI guardrails. Leaders everywhere worry about AI doing things it shouldn’t. This role answers that fear by establishing clarification logic, escalation rules, “never answer” policies, and safety boundaries. The goal is predictable behavior that protects customer trust — an essential pillar of any AI Strategy and AI risk management practice.

4. Aligns reporting with leadership. The AI ops lead reports on resolution rate, CX Score, CSAT, automation coverage, and hours saved — making the economic impact visible. That visibility is a foundational step in any credible customer support ai strategy.

Why this role exists now. AI systems are dynamic and require constant tuning. A small dip in quality quickly becomes an operational issue, and no existing role naturally owns that. When someone does, teams feel the benefit almost immediately.

Knowledge manager — builds and maintains the structured knowledge AI depends on. I hear the same thing from leaders again and again: AI is only as good as the content you give it. This role is rapidly evolving from classic knowledge management into knowledge strategy — part content designer, part systems thinker, part information architect. Their job is to build the knowledge scaffolding that lets AI answer accurately, consistently, and safely.

Here’s how the knowledge manager creates leverage.

1. Writes, maintains, and improves support knowledge — continuously. After every product change, they update articles, remove duplication, resolve contradictions, and pay down “knowledge debt” that quietly erodes accuracy. The upkeep is shaped by AI performance; when patterns expose gaps, they fix the source.

2. Structures knowledge for AI, not for browsing. Traditional help centers are for humans skimming pages. AI needs clean intent signals, crisp formatting, and clearly structured language. The knowledge manager designs that structure as intentionally as the content itself.

3. Works hand-in-hand with AI ops. Many performance issues stem from missing or unclear knowledge. When the AI ops lead surfaces recurring misunderstandings or low-resolution categories, the knowledge manager resolves the root cause at the source.

4. Ensures accuracy and compliance at scale. As AI handles more sensitive situations, the knowledge manager safeguards correctness, currency, and compliance — critical for data governance and regulatory alignment.

5. Develops a cross-functional knowledge strategy. The role creates a canonical, cross-functional source of truth that product, engineering, product marketing, go-to-market, and support (AI and human) can all rely on.

Why this role exists now. This is one of the highest-leverage positions in an AI-first support org. Teams like Rocket Money and Anthropic are hiring knowledge managers because AI accuracy depends on the quality of knowledge feeding it. Without this role, resolution rate caps out early and never climbs.

Conversation designer — designs how the AI speaks, clarifies, and interacts. AI isn’t just a tool customers use; it’s a representative they interact with. Tone, clarity, pacing, and conversational structure matter, especially in voice. Every word affects perceived expertise, trustworthiness, and brand. The conversation designer ensures the AI feels human-friendly without pretending to be human — the sweet spot that builds trust without misleading customers.

In my experience, staffing conversation design early accelerates results. It changes not only how we tune AI, but how we understand the end-to-end customer experience.

Here’s what great conversation design looks like.

1. Shapes the AI’s tone, voice, and communication style. This role refines phrasing, tunes politeness, adjusts how confusion is handled, and shapes micro-interactions that determine whether customers feel cared for or dismissed. On voice channels, natural cadence is make-or-break.

2. Designs flows for high-value conversations. They design how the AI clarifies intent, branches, communicates uncertainty, verifies details, escalates, hands off, and returns to the main thread without feeling mechanical — treating customer experience as a product with language as the interface.

3. Translates procedures and complex workflows into natural language and logic. As AI runs structured procedures and actions, this role becomes a conversational system architect, translating SOPs into conditional logic with exceptions and fallbacks. For example, in Intercom, our conversation designer uses Simulations to run simulated conversations to see where the AI Agent gets confused, over-confident, or awkward, and refine flows until the interaction feels effortless end-to-end.

4. Ensures transitions to humans feel smooth and respectful. Handoffs should provide clear context to the human agent and maintain continuity so customers never feel dropped.

Why this role exists now. As AI becomes the primary interface, conversation design directly influences trust, brand perception, and operational outcomes. It’s a core competency for any Generative AI and LLMs for product managers program.

Support automation specialist — builds the backend actions that allow AI to do real work. If the conversation designer shapes expression, this role shapes capability. They transform AI from an answering machine into an outcome engine by bridging AI and the systems it must safely and deterministically act on.

Support teams increasingly expect AI to do what a human would do: refund a charge, adjust a subscription, verify an identity, update an account setting, or pull relevant data. That expectation creates a new technical role at the edge of support, ops, and engineering.

What I rely on this specialist to deliver.

1. Creates and maintains backend workflows the AI executes. This includes building and maintaining: Fin Tasks. Fin Procedures with embedded steps. Action flows that call internal and external APIs. Automations that span billing systems, user identity layers, CRM objects, subscription entitlements, refund tools, and more. They ensure the AI can act compliantly and predictably — the playbooks that turn intent into action.

2. Owns the integrations required for advanced automation. Many problems require data elsewhere — billing platforms, internal databases, systems of record. The specialist ensures the AI can retrieve, validate, and use that information safely, often partnering closely on CRM integration and internal services.

3. Partners closely with product and engineering. Some workflows require new endpoints, permission layers, safety gates, or deterministic fallbacks. This role drives those changes across the stack.

4. Ensures reliability and safety at every step. Guardrails, validation logic, exception handling, safe execution paths — all are essential. They confirm that the AI has access to the correct data, the action matches policy, edge cases are accounted for, risky flows have deterministic constraints, and every action is auditable and reversible.

Why this role exists now. Customers don’t want answers, they want outcomes. AI can now deliver those outcomes, but only with the right backend scaffolding. This role modernizes operational architecture and unlocks end-to-end automation.

How these roles work together — the new operating loop. These roles aren’t silos; they’re interdependent parts of one system. The AI ops lead identifies patterns and performance gaps. The knowledge manager resolves inaccuracies or missing content. The conversation designer improves clarity, tone, and flow. The automation specialist expands the system’s ability to take action. Each improvement compounds the next, moving you from early automation to transformational resolution rates through continuous refinement.

This loop is what separates teams that plateau early from teams that scale AI into a reliable, high-performing system — the essence of a durable AI Strategy.

How to get started (even if you can’t hire all four roles today). Most teams phase into this model: assign partial ownership, formalize responsibilities, then specialize as AI volume grows. Here’s the progression I recommend.

Phase 1: Assign ownership. Give each role’s core responsibilities to someone who can devote five to 10 hours weekly. Early on, support ops, enablement, senior ICs, and technically inclined teammates can anchor the work.

Phase 2: Formalize the responsibilities. As AI resolves more queries, optimization becomes core operational work. Formalizing ownership prevents performance drift and knowledge debt.

Phase 3: Specialize and hire. Once AI handles 50–70% of incoming volume, these responsibilities become full-time roles. Investing in specialization becomes essential infrastructure for the next scale stage.

The bottom line. AI changes the shape of your support team. These four roles — AI operations lead, knowledge manager, conversation designer, and support automation specialist — form the backbone of the AI-first support organization. They bring order to a constantly changing environment and enable AI to deliver the outcomes leaders and customers expect heading into 2026.

Next week, we’ll continue the 2026 planning series with a deep dive into org design models for AI-first support teams — how to structure people, workflows, and accountability in a world where AI resolves most conversations before a human ever sees them.

To follow along with the series and have each new edition emailed to you directly, drop your details here.

Inspired by this post on The Intercom Blog.

December 2, 2025