I’ve been closing the year with a deliberate reflection ritual for more than a decade, and this season I found fresh energy for it after listening to an insightful conversation with Teresa Torres and Petra Wille on All Things Product. Their approaches mirror the evolution many product leaders experience: moving from rigid annual goal-setting to values-led themes, longer time horizons, and a healthier respect for spaciousness. In my own practice, that shift has created better focus, less pressure, and far more meaningful outcomes.
Prefer to listen? You can find this episode here: Spotify | Apple Podcasts. I took notes with my team in mind and translated the discussion into a simple, values-driven framework that any product organization can adopt.
Why does annual reflection matter for product people? Because our work lives at the intersection of ambiguity, trade-offs, and time. If we only measure ourselves by shipped output or quarterly OKRs, we overlook the compounding value of learning, relationships, and judgement. I treat this ritual as a strategic reset: a chance to surface patterns, adjust expectations, and recommit to outcomes over output.
My own reflection habit started scrappy—paper notebooks, messy timelines, and even artful visualizations inspired by Dear Data by Giorgia Lupi & Stefanie Posavec. Like Petra, I’ve found that tactile, analog artifacts unlock insights I miss in a spreadsheet. Over time, I’ve kept the spirit and simplified the mechanics: a “what went well” review, a short list of hard lessons, and a handful of decisions that paid off—or didn’t.
The biggest evolution for me has been moving from rigid annual goals to values and themes. I still run OKRs, but I use them to track progress, not identity. The lens of process vs. outcome goals—reinforced by ideas from Atomic Habits—helped me set fewer, better commitments. For example, instead of “launch X by Y,” I’ll emphasize the cadence of customer discovery, the health of the product trio, and the quality of decisions made along the way.
One exercise that changed my practice is the “100 wishes” list. It’s powerful—and surprisingly difficult. Pushing past 30 or 40 wishes forces me to name latent interests and long-range intentions I rarely say out loud. Combined with decade-level themes, the list helps me balance ambition with patience. I don’t try to do it all next year; I use it to spotlight direction, not deadlines.
I also review patterns across years: Where did over-scheduling create hidden costs? When did I protect focus time and what did that unlock? Paul Graham’s Maker’s Schedule, Manager’s Schedule remains a useful calibration tool here. And when I feel the pull toward constant throughput, I revisit Stefan Sagmeister’s The Power of Time Off (TED Talk) to remind myself why strategically creating space often yields the most valuable ideas.
Of course, not every year follows plan—and that’s normal. Reflection helps me spot unrealistic expectations early and let them go. When setbacks hit, I’ll rewatch Dealing with Setbacks and re-ground in continuous discovery. The question isn’t “Did we do everything?” but “Did we learn fast, protect customer value, and make trade-offs aligned with our values?” That’s how empowered product teams compound impact.
My sharing philosophy has become more nuanced over time. Some reflections are public to invite dialogue and accountability; others stay private so I can process honestly. I’ve found it helpful to publish what I’m saying no to, capture a theme for the year ahead, and keep the rest for myself and my team. This balance preserves motivation while still contributing to the broader product management leadership community.
If you’re designing your own ritual, consider this lightweight flow: review wins and tough calls, write your “100 wishes,” extract a few values-based themes, then translate those into process goals for Q1. Revisit monthly, not just annually. If you like structured prompts, Chris Guillebeau’s How to Conduct Your Own Annual Review from The Art of Nonconformity offers a practical template you can adapt to your context.
For deeper dives and complementary ideas, I bookmarked these as part of my year-end reset: What I’m Saying No to This Year—And Why, Ask Teresa: My Leaders Still Want Roadmaps with Timelines—What Should I Do?, Scaling Impact: A Look at the Year Ahead (2022), Let’s Connect in 2025: A Look at the Year Ahead, The Interview Coach, and Petra’s own year-ahead reflections (here and her 2026 version). I also recommend revisiting the prior conversation on leadership and change: Role of Leadership in Transformations.
I’d love to hear how you approach your end-of-year reflection. What questions bring you the most clarity? Which practices help you set an intentional, values-driven path for the next year? Share your process—I’m always looking to learn from other product creators and leaders.
I build and scale analytics platforms with a product mindset, and the work starts with the "middleware and compute systems that power analytics at scale." In platforms like Amplitude analytics and other unified analytics platform architectures, that foundation is what makes everything else possible.
Day to day, I oversee the "APIs behind charts, cohorts, and metrics—driving performance, reliability, and platform scalability." When those APIs are fast and resilient, every product team—from growth to customer success—can trust the insights they use to ship, learn, and iterate.
From an engineering leadership standpoint, I partner closely with SRE to define SLOs and error budgets, wire CI/CD pipelines for safe deploys, and track DORA metrics so we improve speed without compromising quality. This combination reduces incident management toil and shortens MTTR while keeping data freshness and query latency within strict thresholds.
From a product management leadership lens, the goal is clarity: crisp APIs, predictable contracts, and transparent stakeholder management across data, engineering, and GTM teams. That alignment empowers product teams with reliable cohorts and metrics, accelerates experimentation, and de-risks roadmaps.
If you’re scaling analytics, invest first in the platform layer: middleware and compute, schema governance, caching strategies, and cost-aware compute. Do that well, and the visible experience—charts, cohorts, and metrics—feels effortless, even as you grow to serve billions of events with confidence.
Inspired by this post on Amplitude – Best Practices.
Your team has more plausible work than capacity. Sales has a customer commitment, support sees recurring friction, engineering sees reliability debt, and executives want a differentiator. Every item can be defended. That is exactly why ranking features is the wrong first move.
An outcome-led system changes what earns priority. You first decide which customer behavior, product condition, or business result needs to change. Then you compare opportunities and solution bets by how credibly they can cause that change. The roadmap becomes a record of choices, evidence, and trade-offs rather than a queue controlled by the loudest request.
Prioritize the change before you prioritize the work
An output is something the team delivers. An outcome is an observable change the team intends to cause. Launching an onboarding flow is an output. Increasing the share of new customers who complete setup successfully is an outcome. The distinction matters because a team can deliver the first without achieving the second.
A usable outcome needs more than a metric name. It should identify who is affected, what behavior or condition should change, why that change matters, how it will be observed, and which guardrails must remain healthy. If you cannot describe how the world should be different after the work succeeds, the item is not ready to compete for priority.
Use an outcome card before accepting solution proposals:
Decision context: the strategic problem that makes a choice necessary.
Target population: the customer segment, user role, or workflow affected.
Current state: the observed behavior, baseline signal, or product condition.
Desired movement: the direction of change and, when the evidence supports it, a meaningful target.
Strategic connection: how the change supports growth, retention, trust, efficiency, or another declared priority.
Guardrails: the signals that must not be harmed while the primary outcome improves.
Review trigger: the evidence or constraint change that would cause leadership to reconsider the outcome.
Do not invent a precise target when no baseline exists. The first commitment may need to be instrumentation, observation, or a small test that establishes the current state. False precision makes an outcome look settled while hiding the most important uncertainty.
The following layers prevent strategy, outcomes, opportunities, bets, and outputs from collapsing into one roadmap item:
Layer
Decision question
Illustrative setup example
Strategic intent
Why does this area matter?
Make first use dependable for new customers.
Outcome
What observable change should occur?
Increase the share of new administrators who finish setup without support.
Opportunity
What unmet need or obstacle prevents that change?
Administrators cannot tell which permissions are required.
Bet
What intervention might address the opportunity?
Test guided permission configuration.
Output
What would the team actually deliver?
Release the validated setup change.
This separation gives you several places to change course. If the bet fails but the opportunity remains important, try another solution. If evidence shows the opportunity was misdiagnosed, investigate another obstacle. If the outcome no longer supports strategy, stop the entire branch. Without these layers, leaders often preserve a feature commitment long after its original reasoning has failed.
A company-level result such as revenue can be valid, but it may be too distant for a product team to manage directly. Connect it to customer behavior and product signals the team can influence. Pair each primary signal with a guardrail: setup completion with setup errors, faster resolution with customer-reported quality, or increased usage with reliability. A metric can improve through the wrong mechanism, so success needs a boundary as well as a direction.
Translate strategy into a decision boundary teams can use
Outcome-led leadership does not mean selecting a metric and disappearing. Leadership owns the strategic context, the outcome boundary, the investment constraints, and the conflicts that individual teams cannot resolve. The team needs room to investigate opportunities, compare solutions, and stop weak bets without asking permission at every step.
Training teams in discovery while leaders continue to manage through feature requests, static roadmaps, and approval gates teaches the organization that customer evidence is secondary. Teams may perform interviews and experiments, but they will still optimize for getting a predetermined feature approved and shipped.
A clear outcome statement can act as a decision boundary:
For [target segment] in [specific situation], improve [behavior or product condition], observed through [primary signal], because [strategic reason], while protecting [guardrails]. Explore opportunities within [scope and constraints] without assuming [requested solution].
The last clause is important. A feature hidden inside an outcome statement is still a feature mandate. Improve adoption of the new dashboard assumes the dashboard is the answer. Help account owners notice and act on performance risks leaves room to discover whether a dashboard, alert, workflow change, or no new interface is the better intervention.
Build a driver tree when the connection between strategy and team behavior is unclear:
Place the business result at the top.
Identify customer behaviors or product conditions that may contribute to it.
Attach observable product signals to those drivers.
Map the customer opportunities that could change each driver.
Mark every unproven connection as an assumption, not a fact.
The tree is not proof of causality. It is a visible model of the current reasoning. That visibility helps teams choose what to validate and helps leaders see where a confident roadmap rests on a weak connection.
Before assigning an outcome, leadership should answer four practical questions:
Why does this outcome deserve investment ahead of the alternatives?
Which constraints are fixed, and which are merely preferences?
Which decisions can the team make without another approval?
What evidence would cause leadership to change the outcome or its investment?
A team cannot genuinely own an outcome when every solution needs executive approval, critical dependencies remain unresolved, or performance is judged only by shipping. That arrangement gives the team accountability without authority. The leadership task is to remove those contradictions before asking the team to move a metric.
Prioritize opportunities with evidence, then shape the portfolio
Use an eligibility gate before a ranking formula
I prefer a gate before a rank. It prevents a polished request with a confident sponsor from competing against a well-understood opportunity merely because both have feature names and effort estimates.
A candidate should become eligible for prioritization only when its decision brief covers:
Outcome relevance: the specific outcome it could affect.
Target evidence: the segment, situation, and observed problem behind it.
Mechanism: the reason this intervention might change the outcome.
Measurement: the primary signal, guardrails, and method of learning.
Critical assumption: the belief most likely to invalidate the bet.
Constraint fit: the technical, operational, and sequencing limits that matter.
Opportunity cost: the work, learning, or outcome investment that would be displaced.
Reversibility: the cost of changing course if the assumption proves wrong.
If a candidate cannot name its outcome or target population, return it to intake. That does not mean it lacks value. It means the organization does not yet have enough information to compare it honestly.
Scoring models can help expose disagreement, but arithmetic should not make weak evidence look objective. Record the reasoning behind each score. Ask which uncertain input has the greatest effect on the ranking. If a small change to that input reverses the decision, investigate the assumption before committing substantial capacity.
Compare opportunities before comparing solutions. Several feature requests may be different guesses about the same customer obstacle. Combining them at the opportunity level can reveal a smaller or more effective intervention. Conversely, two similar-looking features may serve different segments and outcomes, which means one score should not flatten them into a false equivalence.
Use the Kano Model to balance protection, improvement, and exploration
Outcome relevance tells you why an opportunity matters. The Kano Model adds a customer-expectation lens by separating capabilities into must-haves, satisfiers, and delighters.
Must-haves protect the baseline. When they are missing or broken, trust and satisfaction suffer even if the product has innovative features.
Satisfiers create more value as their performance improves. Compare the expected incremental outcome movement with the effort and risk required.
Delighters create unexpected value and differentiation. Treat them as hypotheses worth testing, not as compensation for a broken baseline.
Run the classification by segment and context. A capability can be essential for an advanced customer and irrelevant to a new user. Ask how the target customer would feel if the capability existed and how that same customer would feel if it did not. Pairing these functional and dysfunctional questions is more informative than collecting positive reactions to a proposed feature in isolation.
Do not translate the categories into equal allocations. The right portfolio depends on product maturity, strategic intent, and the condition of the core experience. Make the allocation explicit instead: which investments protect required value, which improve an outcome customers already care about, and which explore future differentiation?
Revisit the classification after meaningful releases or market changes. A delighter can become an expected baseline, so yesterday’s differentiator may no longer justify the same investment. Usage, experiments, interviews, retention patterns, and support evidence should update the portfolio rather than merely confirm the original roadmap.
Run leadership reviews that force choices, not status reports
An outcome-led roadmap can still become output-led in the review meeting. If leaders ask only about delivery dates, scope, and percentage complete, teams will optimize for those signals. Separate the conversations that answer different questions:
Outcome review: Is the customer behavior or product condition moving, for which segment, and with what guardrail effects?
Discovery review: What changed in the team’s understanding of the opportunity, mechanism, or critical assumption?
Commitment review: Which bet should start, continue, change, or stop, and what does that choice displace?
These conversations can share a meeting, but they should not share one vague status label. On track can mean delivery is proceeding to plan while the underlying evidence is weakening. Healthy delivery and healthy product reasoning are different states.
Use a compact review board with the outcome and segment, current signal relative to baseline, strongest new evidence, largest unresolved assumption, active bet, decision required, and displaced work. Feature completion belongs in the delivery portion of the review. It should not stand in for evidence that the outcome is becoming more likely.
Leaders should repeatedly ask:
What did the team learn that it did not believe before?
Which evidence supports or weakens the proposed mechanism?
Is the outcome still right even if the current solution is wrong?
What is the smallest next commitment that resolves the most consequential uncertainty?
What will stop or move if this work receives priority?
Does the team need a decision, a constraint removed, or simply space to continue?
Set decision conditions before attachment to a solution grows. Continue a bet when the evidence strengthens its mechanism. Change the bet when the outcome and opportunity remain valid but the solution does not. Move to another opportunity when the original problem is weaker than expected. Reconsider the outcome when its strategic premise or target segment changes. Stopping a bet is not abandoning outcome ownership; it is one of the ways outcome ownership becomes real.
Stakeholder requests need the same discipline. Translate each requested feature into an intake record that identifies the affected customer, the situation, the observed problem, the evidence, the desired behavior change, the timing constraint, and any alternatives already tried. A request earns evaluation, not an automatic roadmap position.
A useful escalation rule is simple: anyone asking to add committed work must identify what should leave, or explain which outcome or constraint has changed. This turns hidden priority overrides into visible strategy decisions. Seniority may change who has decision rights, but it should not erase opportunity cost.
Before changing the entire organization, use a pilot team to surface decision bottlenecks, incentive conflicts, stakeholder friction, and policy barriers. Track where the team still needs feature approval, where evidence loses to hierarchy, and where another function is rewarded for behavior that undermines the outcome. Those blockers are leadership work. Scaling the workflow without resolving them only distributes the same conflict more widely.
Key takeaways for your next prioritization review
Prioritize an observable customer, product, or business change before ranking proposed outputs.
Give each outcome a target population, baseline signal, strategic connection, guardrails, and review trigger.
Separate outcomes, opportunities, solution bets, and outputs so a failed solution does not preserve itself as a permanent commitment.
Use an evidence gate before scoring, and expose the assumption that could reverse the ranking.
Balance Kano must-haves, satisfiers, and delighters deliberately instead of treating every request as the same kind of value.
Make leadership reviews decide what starts, changes, stops, or gets displaced.
Convert stakeholder urgency into evidence, constraints, and explicit opportunity cost.
At your next roadmap review, take the highest-ranked feature and rewrite it as an outcome statement. Require competing bets to name their evidence, critical assumption, guardrails, and displaced work. If the team cannot do that yet, commit to resolving the uncertainty rather than pretending the feature is ready.
At the following review, ask what changed in the customer signal or the team’s belief before asking what shipped. That question reveals whether your operating system actually rewards outcomes or merely uses outcome language around a feature queue.
AI doesn’t fail because the model is bad, it fails because ownership is missing.
When someone truly owns your AI, everything changes. Resolution and automation rates climb, the system self-improves, and the customer experience transforms in ways a dashboard alone will never show you.
This is part three of our five-part series on customer service planning for 2026. We’ll be sharing all five editions on our blog and on LinkedIn.
If you’d rather have them emailed to you directly as they’re published, drop your details here.
Last week, we introduced the four roles that make AI actually work in a support organization. These roles are already showing up inside the teams who are scaling AI the fastest, and this week, we get closer to the ground.
Here’s what these roles look like in practice — what they do, how they work, and why your AI performance will inevitably drift without them.
AI operations lead — owns AI performance, every day. I think of this person as the air-traffic controller for our AI Agent. I treat the AI as a living system that needs ongoing supervision, evaluation, and tuning. This role is accountable for what leaders care about most: quality, reliability, and continuous improvement.
The AI ops lead sees the whole picture: conversation quality, missing knowledge, flawed assumptions, unexpected failures, new opportunities for automation, and the subtle signals that the system is beginning to drift. In practice, that vigilance is the difference between steady gains and slow decline.
Day-to-day, here’s what I expect from this role.
1. Reviews AI conversations and surfaces performance patterns. The AI ops lead monitors the AI Agent’s behavior — the tone shift after a product launch, a sudden dip in resolution for a specific intent, or conversation clusters revealing new customer behavior. They scan for anomalies, trends, and early warnings, with an emphasis on what’s happening right now, not last week. Without this intentional ownership, I’ve watched a 2% dip turn into a 10% drop in days.
2. Prioritizes fixes and improvements. Once patterns emerge, they triage fixes like a product team handles bugs. Missing or incorrect content? They route it to the knowledge manager. Behavioral issues? They adjust guidance and guardrails. Action or system issues? They partner with the automation specialist. This connective tissue turns individual fixes into compounding improvements.
3. Defines and maintains AI guardrails. Leaders everywhere worry about AI doing things it shouldn’t. This role answers that fear by establishing clarification logic, escalation rules, “never answer” policies, and safety boundaries. The goal is predictable behavior that protects customer trust — an essential pillar of any AI Strategy and AI risk management practice.
4. Aligns reporting with leadership. The AI ops lead reports on resolution rate, CX Score, CSAT, automation coverage, and hours saved — making the economic impact visible. That visibility is a foundational step in any credible customer support ai strategy.
Why this role exists now. AI systems are dynamic and require constant tuning. A small dip in quality quickly becomes an operational issue, and no existing role naturally owns that. When someone does, teams feel the benefit almost immediately.
Knowledge manager — builds and maintains the structured knowledge AI depends on. I hear the same thing from leaders again and again: AI is only as good as the content you give it. This role is rapidly evolving from classic knowledge management into knowledge strategy — part content designer, part systems thinker, part information architect. Their job is to build the knowledge scaffolding that lets AI answer accurately, consistently, and safely.
Here’s how the knowledge manager creates leverage.
1. Writes, maintains, and improves support knowledge — continuously. After every product change, they update articles, remove duplication, resolve contradictions, and pay down “knowledge debt” that quietly erodes accuracy. The upkeep is shaped by AI performance; when patterns expose gaps, they fix the source.
2. Structures knowledge for AI, not for browsing. Traditional help centers are for humans skimming pages. AI needs clean intent signals, crisp formatting, and clearly structured language. The knowledge manager designs that structure as intentionally as the content itself.
3. Works hand-in-hand with AI ops. Many performance issues stem from missing or unclear knowledge. When the AI ops lead surfaces recurring misunderstandings or low-resolution categories, the knowledge manager resolves the root cause at the source.
4. Ensures accuracy and compliance at scale. As AI handles more sensitive situations, the knowledge manager safeguards correctness, currency, and compliance — critical for data governance and regulatory alignment.
5. Develops a cross-functional knowledge strategy. The role creates a canonical, cross-functional source of truth that product, engineering, product marketing, go-to-market, and support (AI and human) can all rely on.
Why this role exists now. This is one of the highest-leverage positions in an AI-first support org. Teams like Rocket Money and Anthropic are hiring knowledge managers because AI accuracy depends on the quality of knowledge feeding it. Without this role, resolution rate caps out early and never climbs.
Conversation designer — designs how the AI speaks, clarifies, and interacts. AI isn’t just a tool customers use; it’s a representative they interact with. Tone, clarity, pacing, and conversational structure matter, especially in voice. Every word affects perceived expertise, trustworthiness, and brand. The conversation designer ensures the AI feels human-friendly without pretending to be human — the sweet spot that builds trust without misleading customers.
In my experience, staffing conversation design early accelerates results. It changes not only how we tune AI, but how we understand the end-to-end customer experience.
Here’s what great conversation design looks like.
1. Shapes the AI’s tone, voice, and communication style. This role refines phrasing, tunes politeness, adjusts how confusion is handled, and shapes micro-interactions that determine whether customers feel cared for or dismissed. On voice channels, natural cadence is make-or-break.
2. Designs flows for high-value conversations. They design how the AI clarifies intent, branches, communicates uncertainty, verifies details, escalates, hands off, and returns to the main thread without feeling mechanical — treating customer experience as a product with language as the interface.
3. Translates procedures and complex workflows into natural language and logic. As AI runs structured procedures and actions, this role becomes a conversational system architect, translating SOPs into conditional logic with exceptions and fallbacks. For example, in Intercom, our conversation designer uses Simulations to run simulated conversations to see where the AI Agent gets confused, over-confident, or awkward, and refine flows until the interaction feels effortless end-to-end.
4. Ensures transitions to humans feel smooth and respectful. Handoffs should provide clear context to the human agent and maintain continuity so customers never feel dropped.
Why this role exists now. As AI becomes the primary interface, conversation design directly influences trust, brand perception, and operational outcomes. It’s a core competency for any Generative AI and LLMs for product managers program.
Support automation specialist — builds the backend actions that allow AI to do real work. If the conversation designer shapes expression, this role shapes capability. They transform AI from an answering machine into an outcome engine by bridging AI and the systems it must safely and deterministically act on.
Support teams increasingly expect AI to do what a human would do: refund a charge, adjust a subscription, verify an identity, update an account setting, or pull relevant data. That expectation creates a new technical role at the edge of support, ops, and engineering.
What I rely on this specialist to deliver.
1. Creates and maintains backend workflows the AI executes. This includes building and maintaining: Fin Tasks. Fin Procedures with embedded steps. Action flows that call internal and external APIs. Automations that span billing systems, user identity layers, CRM objects, subscription entitlements, refund tools, and more. They ensure the AI can act compliantly and predictably — the playbooks that turn intent into action.
2. Owns the integrations required for advanced automation. Many problems require data elsewhere — billing platforms, internal databases, systems of record. The specialist ensures the AI can retrieve, validate, and use that information safely, often partnering closely on CRM integration and internal services.
3. Partners closely with product and engineering. Some workflows require new endpoints, permission layers, safety gates, or deterministic fallbacks. This role drives those changes across the stack.
4. Ensures reliability and safety at every step. Guardrails, validation logic, exception handling, safe execution paths — all are essential. They confirm that the AI has access to the correct data, the action matches policy, edge cases are accounted for, risky flows have deterministic constraints, and every action is auditable and reversible.
Why this role exists now. Customers don’t want answers, they want outcomes. AI can now deliver those outcomes, but only with the right backend scaffolding. This role modernizes operational architecture and unlocks end-to-end automation.
How these roles work together — the new operating loop. These roles aren’t silos; they’re interdependent parts of one system. The AI ops lead identifies patterns and performance gaps. The knowledge manager resolves inaccuracies or missing content. The conversation designer improves clarity, tone, and flow. The automation specialist expands the system’s ability to take action. Each improvement compounds the next, moving you from early automation to transformational resolution rates through continuous refinement.
This loop is what separates teams that plateau early from teams that scale AI into a reliable, high-performing system — the essence of a durable AI Strategy.
How to get started (even if you can’t hire all four roles today). Most teams phase into this model: assign partial ownership, formalize responsibilities, then specialize as AI volume grows. Here’s the progression I recommend.
Phase 1: Assign ownership. Give each role’s core responsibilities to someone who can devote five to 10 hours weekly. Early on, support ops, enablement, senior ICs, and technically inclined teammates can anchor the work.
Phase 2: Formalize the responsibilities. As AI resolves more queries, optimization becomes core operational work. Formalizing ownership prevents performance drift and knowledge debt.
Phase 3: Specialize and hire. Once AI handles 50–70% of incoming volume, these responsibilities become full-time roles. Investing in specialization becomes essential infrastructure for the next scale stage.
The bottom line. AI changes the shape of your support team. These four roles — AI operations lead, knowledge manager, conversation designer, and support automation specialist — form the backbone of the AI-first support organization. They bring order to a constantly changing environment and enable AI to deliver the outcomes leaders and customers expect heading into 2026.
Next week, we’ll continue the 2026 planning series with a deep dive into org design models for AI-first support teams — how to structure people, workflows, and accountability in a world where AI resolves most conversations before a human ever sees them.
To follow along with the series and have each new edition emailed to you directly, drop your details here.
I recently revisited a topic I once pushed back on: using AI to analyze (and maybe even synthesize) customer interviews. After six months of real-world experiments and countless conversations with seasoned product leaders, I’ve evolved my perspective. There is meaningful value here—but only when we’re clear about where AI helps and where it quietly erodes the hard-won customer understanding that powers great product decisions.
If you want to experience the conversation that sparked this reflection, you can listen to the episode on Spotify or Apple Podcast, and watch the discussion here: YouTube. It’s a candid, practical exploration of AI’s role in continuous discovery, and it mirrors what I’m seeing on the ground with product trios and empowered product teams.
Here’s the crux: AI raises the floor for beginners but accelerates experts even more. That matches my experience—early-career PMs get structure, momentum, and a confidence boost, while experienced interviewers can move faster without sacrificing nuance. But there’s a catch. If your interviewing skills aren’t solid yet, AI can create a veneer of insight that masks shallow understanding. In other words, it can help you go wrong more efficiently.
The conversation makes an important distinction between analysis and synthesis. Analysis is about extracting signals from the interview. Synthesis is about building meaning—connecting patterns, weighing contradictions, and deciding what to do next. AI can speed up the former with summaries and highlights. The latter—true synthesis—still demands expert judgment, context, and empathy.
One line from the episode stuck with me: your unpolished interview skills matter more than any shiny new AI workflow. I’ve felt that firsthand. When interview quality is uneven, dropping transcripts into an LLM won’t save you. You still need to synthesize every interview individually so the signals remain traceable and credible. That discipline keeps teams aligned, prevents overfitting to noise, and builds the organizational memory that fuels better bets.
We also explored the operational reality most teams face: interviews pile up. Backlogs grow. Leaders want speed. This is where “expert + AI” shines. With the right prompts, templates, and context, tools like ChatGPT and Claude can help transform raw transcripts into structured artifacts you can trust—provided a strong interviewer sets the frame and makes the calls. That balance preserves both velocity and quality.
What changed my mind most was the evidence from experiments—running sets of interviews through different LLMs and comparing outcomes. The patterns were consistent: beginner + AI is usually better than nothing, but the real performance gains come from expert + AI. When experts guide the process, AI becomes an accelerant rather than a crutch.
A favorite story in the episode takes a detour into building a gaming PC—an unexpected but perfect metaphor for AI’s limits. You can get great step-by-step guidance from a model, but when context shifts or edge cases appear, expertise is what keeps you from making expensive mistakes. Customer interviews are like that. Empathy comes from human interaction; AI can’t replace the experience of talking directly to your customers.
My practical guidance for teams integrating AI into continuous discovery: start with interviewing fundamentals, separate analysis from synthesis, and standardize how you capture single-interview learnings. If you need a tight template for this, refer to “The Interview Snapshot: How to Synthesize and Share What You Learned from a Single Customer Interview.” Use AI for summaries, clustering, and draft artifacts—but have an expert finalize the narratives, evaluate trade-offs, and document assumptions.
If you’re scaling this across an organization, invest in training first, then in workflows. Build a lightweight operating system for discovery: consistent interview guides, “story-based” techniques, and a shared library of prompts. Consider resources like “The Interview Coach,” as well as practical write-ups such as “Customer Interview Analysis: Where AI Helps and Hurts.” These help teams avoid common pitfalls and make better use of AI in high-judgment moments.
My bottom line: AI isn’t magic. It can help, but only if your interviews are strong and you provide the right context. Customer understanding is a competitive moat; outsourcing it entirely will cost you in the long run. Use AI to accelerate—not replace—the human judgment that makes product discovery work.
Resources and links worth exploring: ChatGPT, Claude, The Interview Snapshot: How to Synthesize and Share What You Learned from a Single Customer Interview, The Interview Coach, and Customer Interview Analysis: Where AI Helps and Hurts.
I’d love to hear how your team is using AI in discovery. What’s working, what’s risky, and where do you draw the line between automation and judgment? Share your experiences in the comments—our community learns faster when we compare notes.
When I push our organization to adopt the product operating model, I’m emphasizing a foundational shift—from “shipping roadmaps of features (output)” to solving real customer and business problems, measured by “business results (outcomes)”. That’s the difference between activity and impact, and it’s the only way to build durable value at scale.
This change inevitably reaches beyond the product organization. It reshapes how company stakeholders in Sales, Marketing, Customer Success, Finance, Legal, Security, and Operations engage with product teams, and it reframes what they expect from us. Instead of asking, “When will feature X ship?” they learn to ask, “How will we move the outcome that matters?”
In practice, the product operating model is a contract: product teams commit to outcomes, and stakeholders commit to partnership. That partnership means we co-own the problem, align on evidence, and share accountability for results. The reward is clarity—everyone sees how their work ladders to strategy and why the sequence of work makes sense.
Here’s how I align stakeholders around this model. First, I ground everything in outcomes vs output OKRs. We replace feature roadmaps with a clear strategy, prioritized problems, and measurable objectives. Our product roadmapping and sprint planning then serve the objectives—not the other way around—so capacity is allocated to the highest-leverage bets.
Second, I build empowered product teams around product trios (product, design, engineering). We practice continuous discovery with stakeholders: we share opportunity trees, test riskiest assumptions early, and bring partners into research when it informs go-to-market strategy, pricing, or enablement. This keeps us honest and avoids late-stage surprises.
Third, I establish operating rhythms that make outcomes visible. Monthly stakeholder reviews focus on progress toward objectives and what we’re learning—not status theater. Quarterly, we connect OKRs to business performance so leaders can see the throughline from discovery and delivery to pipeline, retention, or margin. If priorities shift, we renegotiate objectives explicitly.
Fourth, I define metrics that stakeholders trust. We use a balanced set of leading indicators (activation, engagement, cycle time) and lagging indicators (revenue, retention, unit economics). We socialize definitions early so no one debates the scoreboard mid-game. The result: faster decisions and less “data whiplash.”
Fifth, I invest in change management. Moving from outputs to outcomes can feel threatening if your success has historically been measured by launch volume or roadmap commitments. I address this head-on with training, transparent comms, and clear decision rights. The message is simple: outcomes create more autonomy for empowered product teams and more predictability for stakeholders.
At HighLevel, this approach has been especially powerful when cross-functional dependencies are high. For example, when we set an objective to improve user activation for a new CRM integration, we didn’t promise a bundle of features. We committed to a measurable lift in activation and a shorter time-to-value, co-owned with Customer Success and Marketing. That alignment unlocked smarter experiments, tighter enablement, and a more credible launch narrative.
The anti-patterns are predictable: treating OKRs as a renaming of the roadmap, equating discovery with indecision, or isolating product decisions from go-to-market strategy. The cure is equally consistent: bring stakeholders into discovery, attach every bet to an objective, and show progress with evidence—not just demos.
Ultimately, the product operating model is a leadership choice. It asks us to trade certainty theater for learning velocity, and feature checklists for business impact. When stakeholders see that shift pay off—in faster cycles, clearer priorities, and results that matter—support for the model moves from compliance to conviction.
By 2026, the AI Product Owner will be the keystone role that turns AI strategy into measurable business outcomes. In my teams, this seat bridges market insight, model capability, data governance, and shipping velocity—so product decisions are not just clever, but compliant, reliable, and fast.
I often describe the remit simply: "Here is your clear guide to the AI product owner role (skills, responsibilities, how it differs from PM) and ways AI tools supercharge delivery." In practice, the AI Product Owner translates business goals into model-backed experiences, aligns cross-functional execution, and ensures the product’s AI behavior remains safe, lawful, and on-brand under real-world constraints.
How does this differ from a traditional PM? While Product Management sets portfolio strategy, positioning, and market narratives, the AI Product Owner owns the AI experience end-to-end—data readiness, evaluation harnesses, safety guardrails, and the iterative model improvements that drive outcomes vs output OKRs. I anchor the role inside empowered product teams and product trios (PM/Design/ML Eng) to keep discovery continuous and delivery disciplined.
On responsibilities, I expect four pillars. First, discovery: continuous discovery with customers and internal experts to uncover use cases where generative AI or LLMs beat the status quo. Second, experience: define the right interaction patterns for AI UX, including retrieval-first pipeline choices, context window management, and feedback loops for human-in-the-loop correction. Third, governance: privacy-by-design, AI risk management, data governance, and regulatory compliance baked into the roadmap. Fourth, delivery: CI/CD for models and prompts, observable evaluation with A/B testing and minimum detectable effect (MDE), and SRE-grade incident management when AI behavior drifts.
Skills-wise, I look for product sense plus technical fluency. That includes LLMs for product managers (prompting, grounding, RAG), analytics mastery (Amplitude analytics, retention analysis, activation metrics), and comfort with DORA metrics and deployment frequency to keep iteration high but safe. Strong stakeholder management and clear writing are non-negotiable—AI capabilities evolve fast, and leaders must see risk, cost, and ROI with no ambiguity.
AI tools truly supercharge delivery when they eliminate bottlenecks. My practical stack: an AI product toolbox with Claude Code and a ChatGPT connector for rapid prototyping; CustomGPT workflows for support triage and internal knowledge; Pendo product tours and in-app guides to validate behavior changes; Intercom for customer support ai strategy; and tight CRM integration via HubSpot to measure revenue impact. The outcome is faster idea-to-learning cycles, sharper telemetry, and far cleaner handoffs.
For roadmapping, I prioritize thin slices that prove value early—shipping narrowly scoped assistants or copilots, then expanding with product roadmapping and sprint planning that ties capability unlocks to outcomes. A unified analytics platform helps compare human-only baselines to augmented workflows, while agentic AI patterns automate routine steps under strict guardrails.
Risk is a product surface, not a side task. I require explicit policy gates (PII handling, red-teaming, bias audits), clear escalation paths, and incident playbooks. When we treat policy and reliability as features, customers reward us with deeper adoption and higher trust.
If you’re pursuing the AI Product Owner path, build a portfolio around shipped learnings: the experiment you killed with data, the safety constraint you designed, the postmortem you led, and the business metric you moved. That story—evidence of disciplined discovery, responsible delivery, and real-world results—is exactly what teams (and boards) want to see in 2026.
If your community of practice needs constant reminders, fills its agenda with updates, and produces little that teams use afterward, the problem probably is not motivation. The community was given a meeting cadence before it was given a job.
Your job as a product leader is to create a repeatable path from a live problem to a better decision, a stronger practice, and knowledge another team can reuse. That is how you design continuous learning as a system instead of hoping it emerges from another recurring call.
Give the community a practice to improve, not a topic to discuss
A broad subject can attract interest without changing anyone’s work. Product strategy, discovery, AI, leadership, and experimentation are all reasonable areas of interest, but each is too large to serve as an operating purpose.
Start with a practice that members perform and can inspect. Opportunity framing is a practice. Writing an AI evaluation plan is a practice. Preparing an experiment decision is a practice. Stakeholder management is still too broad until you identify the behavior you want to improve, such as exposing trade-offs before a roadmap commitment is made.
A useful purpose statement has four parts:
Members: Who needs to learn together?
Practice: What recurring part of their work should get better?
Learning activity: What will they examine, attempt, or critique together?
Work consequence: What should change in a decision, artifact, or team behavior?
For example: This community helps product trios improve opportunity framing by critiquing active discovery artifacts, so teams can separate evidence from assumptions before choosing a solution.
That statement is narrow enough to guide an agenda. It tells members what to bring, tells a facilitator what kind of discussion belongs, and gives a sponsor something more meaningful to inspect than attendance.
Choose a quarterly learning theme with these filters:
Members are encountering the problem in current work, not merely expressing general interest in it.
The practice is shared enough that one person’s case can teach something useful to others.
A real artifact can make the practice visible. That might be an opportunity map, discovery plan, evaluation set, experiment brief, decision record, or stakeholder narrative.
Improvement can be noticed in later work. You should be able to point to a changed question, assumption, method, trade-off, or decision.
The theme is narrow enough to defer adjacent subjects. A community without boundaries becomes an internal conference with no coherent learning loop.
Write those choices into a short charter. Include the theme, target practice, current definition of good, artifact members will examine, evidence of progress, and what is out of scope. Treat the definition of good as a starting hypothesis. Learning can reveal a stronger standard after the work begins; the charter should be stable enough to focus the community but not so rigid that it prevents that discovery.
Combine learning from people with learning with people
A community needs external input and collaborative practice. Input without practice becomes content consumption. Collaboration without input can recycle the same local assumptions. Design both modes deliberately.
Learning mode
Use it when you need
Useful inputs
Expected output
Learning from people
Depth, a reference point, or a clearer definition of good
A tightly curated personal learning network, talks, books, courses, examples, and practitioners whose decisions you can examine
A heuristic, annotated example, sharper question, or alternative approach to test
Learning with people
Feedback, accountability, new patterns, or pressure-testing
Peer circles, artifact critiques, hackathons, meetups, and cross-functional working sessions
A revised artifact, changed decision, new experiment, or reusable lesson
The bridge between the two modes matters more than the volume of material consumed. Begin with a live question from the work. Curate external input that can sharpen that question. Bring the work artifact to peers. Critique its assumptions and trade-offs. Record what changed. Store the lesson where the next person facing the problem can retrieve it.
For an AI product community, the live question might concern an evaluation plan for a support agent. External examples can help the group notice missing failure cases, but reading alone does not improve the plan. Members need to inspect the proposed evaluation set, challenge what it represents, identify gaps, and document the resulting change. The work becomes the learning surface.
Track the network as working infrastructure. For each person or resource, note the practice you are learning, the artifact or decision that demonstrates it, the question it helps answer, and the action you intend to try. Prune the list when the theme changes or an input repeatedly fails to affect your thinking. The goal is not to follow everyone worth knowing. It is to make the right expertise retrievable when a decision needs it.
Build a cadence that ends in changed work and reusable artifacts
A community meeting is only one step in the learning loop. If the loop begins with an agenda and ends when the call finishes, members may enjoy the conversation while the organization loses most of its value.
A lightweight operating model can fit alongside product delivery:
Set a quarterly theme. Tie it to a practice teams currently need to improve.
Curate a small learning network. Gather examples and perspectives that challenge the community’s current standard.
Run monthly critiques. Use current work from product, design, and engineering rather than hypothetical exercises.
Publish one teaching artifact. Turn the strongest learning into a talk, guide, workshop, template, annotated example, or decision pattern.
Close the loop. Write down what changed in a decision, discovery cadence, product bet, or working method.
Do not ask a member to present everything they know about the theme. Ask them to bring something unfinished that matters to a real decision. The critique should answer a small set of questions:
Decision: What decision is the owner preparing to make?
Artifact: What document, model, prototype, dataset, or plan exposes the current thinking?
Evidence: What is known, what is assumed, and where is confidence weak?
Trade-off: Which constraint or competing objective makes the decision difficult?
Critique request: What does the owner want peers to challenge?
Change: What will the owner revise, test, reject, or investigate after the session?
The final question prevents critique from dissolving into commentary. Advice is not yet learning. Learning becomes visible when the owner changes an artifact, runs a test, revises a decision, or explains why the critique did not alter the course.
Keep the feedback about the work, not the person’s competence. Sensitive examples can be anonymized, but stripping out every constraint makes the exercise artificial. Preserve the decision context, evidence, and trade-offs that peers need in order to give useful criticism.
Separate community roles so the founder is not the system
A community becomes fragile when one enthusiastic leader selects every topic, provides every answer, facilitates every discussion, and writes every note. Distribute the work:
Steward: Maintains the charter, boundaries, and relationship to organizational priorities.
Curator: Finds relevant people, examples, and learning inputs for the current theme.
Facilitator: Keeps sessions focused on the stated decision and critique request.
Artifact owner: Brings live work and decides what to do with the feedback.
Synthesizer: Captures the reusable lesson, change made, and retrieval metadata.
A small community can combine roles, but the responsibilities should still be explicit. Rotating artifact ownership also prevents the group from becoming an expert’s help desk. Members learn to expose their reasoning, offer precise critique, and teach what they have understood.
Use the same structure for every durable artifact: context, decision, evidence, critique, change, result still to be observed, and reusable principle. Tag it by practice and decision type rather than only by meeting date. A folder full of chronological notes is an archive. A collection organized around future retrieval is a knowledge system.
Diagnose failure modes and show evidence of impact
Community leaders often respond to weak participation by adding speakers, reminders, or more topics. Those actions can increase activity while preserving the design flaw. Read the symptom as evidence about the operating model.
What you notice
Likely design problem
What to change
Sessions become status updates
Live work is being reported rather than examined
Remove the progress round. Require a decision, artifact, and explicit critique request.
Conversations are energetic but nothing changes afterward
The learning loop ends at discussion
Close every critique with a named change, test, investigation, or reason for retaining the current approach.
The same experts do most of the talking
The community has become a help desk or lecture series
Rotate artifact ownership and ask members to expose their judgment, not just request answers.
Every session covers a different subject
The theme is too broad or absent
Return to one quarterly practice and place adjacent requests in a backlog.
Notes accumulate but are rarely reused
Capture is organized around meetings rather than retrieval
Use a common artifact template and tag lessons by practice, decision, and problem.
People attend but stop bringing unfinished work
Critique may feel unsafe, performative, or disconnected from current decisions
Review the invitation, keep feedback about the artifact, and let owners state the feedback they need.
The community depends on its founder
Operational knowledge and authority have not been distributed
Make roles explicit, rotate them, and document the cadence.
Do not make attendance your primary success measure. Attendance can show reach, but it cannot tell you whether anyone learned, changed a practice, or made a better-informed decision. It is possible to fill every session and still run a content club with no operational effect.
Use an evidence chain that a product or executive sponsor can inspect:
Participation: Members bring relevant work and a real decision question.
Artifact change: A plan, model, evaluation, narrative, or discovery artifact is revised after critique.
Practice change: A team adopts, tests, or deliberately rejects a method with its reasoning recorded.
Knowledge reuse: Another person can find the artifact and apply it to a later decision.
Decision trace: The close-loop note identifies what changed in the team’s cadence, choices, or bets.
This chain is more defensible than claiming the community directly produced a business outcome. Product teams still own delivery and results. The community improves the quality and availability of the practices those teams use. Connect it to business impact when the trace is real, but do not skip the intermediate evidence.
At the end of the quarterly theme, review the artifacts and ask: Which critiques changed work? Which lessons were reused? Which assumptions survived testing? Which part of the definition of good became clearer? Which unresolved practice deserves the next theme? If you cannot answer those questions, adjust the design before adding another meeting.
Key takeaways
Define the community around a recurring practice and a visible change in work, not a broad topic or an attendance goal.
Combine curated learning from people with artifact-based learning alongside peers.
Use a quarterly theme, monthly critique, teaching artifact, and change record to complete the learning loop.
Make unfinished work the center of each session and end with a revision, test, investigation, or explicit decision.
Organize knowledge for retrieval by practice and decision type, not merely by meeting date.
Show impact through artifact changes, practice changes, reuse, and decision traces before connecting the community to business results.
Before scheduling the next session, write the purpose sentence and name the artifact members will examine. Invite them to bring a live decision, then publish a short record of what changed after the critique. If you cannot name the practice or the expected output yet, keep designing the community before you create its calendar.
I’ve spent years helping talented engineers explore what’s next when pure coding no longer feels like the only—or best—path. From hiring across cross-functional teams to mentoring career pivots, I’ve seen firsthand how engineering strengths translate into high-leverage roles that shape product, strategy, and growth.
Software engineers have alternative career options leveraging their skills in roles like product manager, data scientist, business analyst, and 22 more.
When an engineer moves into product management, they’re not starting from scratch—they’re redirecting problem-solving, systems thinking, and customer empathy toward outcomes. In practice, that means mastering product discovery, strengthening stakeholder management, and getting fluent in product roadmapping and sprint planning, so decisions are guided by impact rather than “outputs vs outcomes” confusion. I’ve watched this transition unlock empowered product teams and clearer prioritization across complex backlogs.
Data-oriented paths are equally compelling. If you enjoy experimentation and evidence-based decisions, roles in analytics or data science reward rigor. Think A/B testing, identifying the minimum detectable effect (MDE), and using tools like Amplitude analytics to translate behavioral signals into product bets. Pair that with retention analysis and you’ll become indispensable to growth conversations.
Business-facing roles such as business analyst or product marketing manager are ideal if you’re energized by customer problems and market narratives. Your engineering fluency sharpens value propositions, product positioning, and go-to-market strategy in a way that resonates with both buyers and builders. In my teams, the best bridges between product and revenue often came from former engineers who could articulate trade-offs with clarity.
If operational excellence is your edge, consider SRE, DevOps, or cybersecurity. The same instincts that push you toward clean CI/CD pipelines and resilient architectures translate well into incident management, threat detection and response, and privacy-by-design practices. These roles reward systems thinking and the ability to balance reliability with delivery speed.
For engineers who love community and storytelling, developer evangelism is a natural fit. You’ll translate complex concepts into actionable guidance, from in-app guides and product tours to UX writing and documentation. The best evangelists I’ve worked with turn feedback loops into product insight, strengthening activation and product-led growth without heavy sales pressure.
Customer-facing technical roles—solutions engineer, forward deployed engineer, or technical consultant—let you stay close to the product while solving real-world problems. You’ll drive onboarding quality, user activation, and adoption while surfacing insights that influence roadmaps. Done well, this work tightens the loop between customer outcomes and product decisions.
AI-centered roles are expanding rapidly. If you’re curious about AI Strategy, retrieval-first pipelines, or the practical use of LLMs for product managers, you can bring an engineer’s discernment to a noisy space. The most valuable contributors here pair pragmatic architecture choices with clear risk management and measurable business value, not hype.
Leadership tracks remain a strong option too. The IC to manager transition isn’t about title; it’s about raising the ceiling for others. You’ll coach empowered product teams, shape organizational development, and align initiatives to defensible metrics—think DORA metrics for flow, leading indicators for value, and OKRs that measure outcomes over output.
If you’re exploring a pivot, start small and intentional. Run “career A/B tests” by taking on cross-functional projects, shadowing adjacent roles, or shipping a lightweight portfolio that demonstrates the new muscle. Join a ProductCon session, practice conference networking, and refine a narrative that links your engineering foundation to the outcomes your target role owns.
Finally, map your personal unfair advantages—domain knowledge, systems thinking, customer empathy, or operational rigor—to the roles that value them most. With focus, you can reposition your engineering experience into a differentiated story that accelerates your next chapter. The breadth of options is real, and with a deliberate plan, you’ll turn curiosity into conviction—and conviction into impact.
Every week, I’m in conversations with product leaders, engineers, and security teams who are trying to ship AI features faster without compromising trust. The tension is real: stakeholders want velocity, customers want transparency, and regulators want accountability. That’s exactly where modern data governance earns its keep.
New AI pressures are redefining what good governance takes. Learn how to build better frameworks, move fast with confidence, and keep your data from being a black box.
In my role leading product management, I’ve learned that robust data governance isn’t a compliance checkbox—it’s a strategic capability. When we treat governance as a product, we architect for clarity, safety, and speed. That means aligning AI Strategy with day-to-day delivery so teams know what they can ship, when, and why.
Here’s the practical blueprint I rely on. First, establish ownership and a shared language. Create a living data catalog, lineage maps, and clear data classifications so teams know which assets are sensitive, regulated, or eligible for training LLMs. Second, harden privacy-by-design and least-privilege access. Bake PII detection, secrets management, and role-based policies directly into your workflows. Third, bring quality and observability to the forefront: instrument data contracts, monitor drift, and track model performance across environments. Finally, implement model governance end to end—dataset cards, model cards, bias testing, human-in-the-loop review, and a repeatable evaluation harness.
To move fast with confidence, make governance invisible and automated. Treat policies as code in CI/CD, gate deployments with pre-merge checks, and fail builds that violate data contracts. Log prompts and outputs responsibly, route unsafe patterns to red-teaming, and use a retrieval-first pipeline to anchor models on verified sources rather than fragile context stuffing. This is how we scale AI product development while keeping audit trails complete and costs in check.
Avoiding the black-box problem starts with transparency. Document assumptions, training data sources, and known limitations—then expose explanations where it matters in the product experience. Pair this with a unified analytics platform to tie telemetry, feature flags, and user feedback to model changes. When something goes sideways, your observability, incident management playbooks, and threat detection and response processes should make root-cause analysis fast and defensible.
If you’re building your program from scratch, use a 30-60-90 approach. In the first 30 days, inventory systems, classify data, and map high-risk use cases. By day 60, formalize RACI for governance, deploy access controls, and set up your evaluation pipeline with golden datasets and measurable acceptance thresholds. By day 90, operationalize incident response, conduct tabletop exercises, and wire governance outcomes into OKRs—think time-to-approval for high-risk changes, reduction in production incidents, and model evaluation pass rates.
This playbook pays off in board conversations and with customers. You can articulate your AI risk management posture, show measurable progress on regulatory compliance, and demonstrate how governance accelerates—not hinders—delivery. Most importantly, your teams gain the confidence to experiment, knowing there’s a safety net that protects users, the brand, and the business.
If your organization is wrestling with how to balance innovation and control, start small, codify what works, and scale with intent. With the right foundations in data governance, AI becomes an engine for durable advantage—not a source of sleepless nights.
Inspired by this post on Amplitude – Perspectives.
What if your morning started with a helpful check-in from a voice AI that actually improves your sleep—using the same core principles that typically cost thousands of dollars and come with year-and-a-half waitlists? That idea energizes me as a product leader, because it blends clinical-grade outcomes with consumer-grade accessibility. Recently, I dug into how the team at Rest built an AI sleep coach inspired by Cognitive Behavioral Therapy for Insomnia (CBTI), and why their method offers a repeatable blueprint for complex, personal AI products.
The origin story is a classic product discovery moment. Rest’s team noticed that a meaningful slice of users in their podcast app were using audio to fall asleep. Although it represented only about 10% of users, that group showed a high willingness to pay. That signal pushed them to explore a dedicated sleep solution, moving from a general audio app to a targeted sleep experience—and eventually toward an AI-powered coach as LLMs matured.
Through jobs-to-be-done research, they identified a clear, underserved segment: “DIY sleep hackers.” These are motivated users who want agency, structure, and results without navigating clinical systems. Choosing CBTI (a clinically proven approach with 80% efficacy) gave the product a strong evidence-based foundation while remaining accessible as a wellness tool. It’s the kind of strategic choice I look for: credible, measurable, and aligned with user motivation.
The product evolution moved in smart, incremental steps. Rest started with a basic text chatbot before graduating to a voice-first experience—using Vapi for voice and OpenAI for reasoning. Voice changed the relationship dynamic: it increased intimacy, lowered friction for daily check-ins, and made behavioral coaching feel human without pretending to be. The team built a memory system that tracks context (like traveling or having a dog) with time-based relevance, which keeps conversations fresh, respectful, and genuinely personalized.
Daily engagement is driven by dynamic agendas that adapt based on sleep data, the user’s stage in the program, and their recent compliance. I love this mechanic: it operationalizes behavior change by sequencing the right intervention at the right time. In parallel, they developed text via OpenAI Assistants while building voice with Vapi, which let them ship value while learning in two modes. They also moved from massive system prompts to RAG for general sleep knowledge, keeping personal user context in the prompt—reducing brittleness while improving scalability.
Because sleep sits close to healthcare, the team drew a firm line between wellness and medical positioning. They implemented clear guardrails: no diagnosis, no medication advice, and strong boundaries on scope. Weekly error analyses with domain experts (sleep therapists) tightened quality and tone, and they adopted LLM-powered evals to enforce safety boundaries. For observability and evaluations, they leveraged Langfuse, and they experimented with Hamming for voice testing to refine the experience end-to-end.
Under the hood, this is a great example of “one bite of the apple at a time” product building in AI. Start with a simple interface, anchor on an evidence-based method, layer personalization with memory, formalize program structure with dynamic agendas, and shift to RAG when general knowledge outgrows prompt engineering. As a product leader, I see strong echoes of agentic patterns here—goal-oriented orchestration, stateful memory, and adaptive planning—shipped in pragmatic increments rather than as a monolithic platform rewrite.
A few takeaways I’m applying with my teams: First, segment deeply and pick a high-intent niche (those “DIY sleep hackers” were the right beachhead). Second, let modality fit the job—voice is not a gimmick when it boosts compliance and empathy. Third, design safety and scope from day one if you’re anywhere near health. Finally, invest early in evals and observability so you can improve with confidence, not hope.
If you want to explore the full conversation and product decisions, you can listen here: Spotify | Apple Podcasts.
Resources & Links:
Rest – AI sleep coach app
Vapi – Voice agent platform Rest uses
Langfuse – Observability and evals platform
Hamming – Voice testing platform
AI Evals Maven Course by Hamel Husain and Shreya Shankar
Bottom line: Rest demonstrates how to take a clinically grounded method like CBTI, translate it into a daily voice-first experience, and ship it with rigor. If you’re building in AI, this is a model worth studying—practical, safe, and deeply user-centered.
Every breakthrough we ship in AI reinforces a simple truth I live by: "Companies that prioritize data quality, governance, and structure will accelerate their AI initiatives the fastest." That statement captures the difference between flashy demos and durable, scalable products. In my experience, the strongest AI Strategy starts with the discipline to treat data as a product, not an afterthought.
When teams rush to production with generative AI or LLMs, the first issues rarely come from the model itself—they come from the data. Poor lineage leads to hallucinations, inconsistent schemas inflate costs, and weak access controls erode trust. For LLMs for product managers, this is the gap between a compelling prototype and a reliable system customers depend on every day.
Let me clarify what I mean by data quality, governance, and structure. Quality is completeness, accuracy, freshness, and consistency across sources. Governance is policy, ownership, and accountability—privacy-by-design, regulatory compliance, and AI risk management built in from day one. Structure is the architecture: clear data contracts, standardized schemas, metadata and lineage, and role-based access that keeps sensitive signals protected while enabling speed.
Here’s the product playbook I use to operationalize this. First, map critical sources and define data contracts at the edges so producers and consumers can move independently. Second, standardize schemas and entity resolution to eliminate ambiguous joins. Third, enforce privacy-by-design with policy-as-code and automated redaction. Fourth, converge analytics into a unified analytics platform so definitions, freshness, and observability are shared. Fifth, instrument end-to-end lineage and quality SLAs with alerting. Finally, close the loop with human feedback and labeling to continuously improve model performance.
For generative AI workloads, a retrieval-first pipeline is essential. Unify trusted sources (product analytics, CRM, support, docs), embed and index them with guardrails, and focus on context window management to keep prompts lean, relevant, and cost-effective. This approach improves response quality, reduces token spend, and makes updates near-real-time—without retraining the base model every week.
Measure what matters. Tie model outcomes to product metrics through rigorous A/B testing, and size experiments with minimum detectable effect (MDE) so you can ship confidently. Use product analytics to verify that better data actually improves activation, retention, and support deflection. When teams can trace an AI improvement back to a specific data-quality fix, they invest in governance with conviction.
Culture closes the gap. Empowered product teams and product trios (PM, design, engineering) make crisper decisions when data stewards are embedded and accountable. Clear ownership, shared definitions, and transparent dashboards reduce friction with security and compliance while speeding up delivery. This is how product management leadership sustains velocity without trading away trust.
The bottom line: if we want faster, safer, and more scalable AI, we start with the data. Build strong foundations, treat governance as enablement, and structure every step so improvements compound. With that in place, Generative AI stops being a science experiment and becomes a durable competitive advantage.
Inspired by this post on Amplitude – Perspectives.
