Category: Product Management Leadership

  • How Product Leaders Should Plan Their 2026 Conference Calendar

    How Product Leaders Should Plan Their 2026 Conference Calendar

    Your 2026 conference budget should not begin with a list of famous events. It should begin with a decision your product organization is struggling to make: where AI belongs in the roadmap, why discovery is not changing priorities, how product operations should reduce friction, or which growth problem deserves executive attention.

    That shift turns conference planning from a travel exercise into a portfolio of strategic bets. You can choose events for the decisions, relationships, and operating changes they can support – and decline the ones that offer interesting content without a credible path to action.

    Decide what each conference must change

    There is no universally best product conference. The right choice depends on the uncertainty you need to reduce and what you are prepared to do with the answer.

    The real cost is larger than registration and travel. It includes the attendee’s attention, the decisions someone else must cover, the interruption to active work, and the follow-through required after the event. An inexpensive ticket can therefore be a poor investment, while a more demanding trip can be defensible when it helps unblock a consequential product decision.

    Before approving a booking, require a short conference thesis with these fields:

    • Decision: What active product, AI, growth, hiring, or operating-model decision should improve?
    • Uncertainty: What does the team not know well enough to decide confidently?
    • External value: Which practitioners, perspectives, or examples are difficult to access inside the company?
    • Action: What will the attendee recommend if the current hypothesis becomes stronger, weaker, or more conditional?
    • Owner: Who has the authority to act on what is learned?

    For an AI product leader, a useful thesis might be: I need operator evidence that helps us decide whether the next investment belongs in a customer-facing capability, an internal workflow, or the evaluation and governance layer beneath both. That is specific enough to shape session choices and conversations. Learn more about AI is not.

    The same standard applies to networking. Meet product leaders is too vague to guide behavior. Compare how product executives assign decision rights between a central AI platform group and embedded product teams gives the attendee a real question, a relevant peer profile, and a reason to follow up.

    Key takeaways

    • Choose a conference for an active decision, not for its reputation alone.
    • Build a portfolio across strategic priorities instead of sending everyone to similar general-interest events.
    • Use date collisions to force explicit tradeoffs about content, peer access, and geography.
    • Send the person closest to the uncertainty, provided that person has enough authority to apply what they learn.
    • Measure changed decisions and operating behavior, not notes collected or badges scanned.

    Build a portfolio around strategic priorities

    Conference names are useful screening signals, but they are not proof of agenda quality. Start by matching event themes to company priorities. Then inspect the organizer’s current agenda, speakers, attendee profile, format, and location before committing money or executive time. Verify the date and venue directly before purchasing because event details can change.

    The following map helps narrow the field. It is a planning tool, not a ranking.

    Strategic priority2026 events to investigateQuestion that should govern selection
    AI products and intelligent interfacesAI Product Summit on April 15 in San Jose, ProductCon AI on August 5 as a virtual event, and ACM IUI from March 23-26 in PaphosDo you need product operating practices, broad virtual access, or deeper thinking about intelligent user interfaces?
    Executive product leadershipGartner Product Leadership Conference on March 9-10 and Chief Product Officer Summits in New York, Palo Alto, Amsterdam, and San FranciscoWhich leadership decision needs calibration with peers who have comparable scope and accountability?
    Product operationsProduct Operations Summits in New York on March 26-27, Amsterdam on May 12-13, and San Francisco on September 22-23Are you trying to improve decision flow, planning, discovery infrastructure, tooling, or cross-functional accountability?
    Product-led growthProduct-Led Summits across Washington, Austin, New York, Denver, Amsterdam, Seattle, London, San Francisco, Berlin, Sydney, Boston, and TorontoWhich growth problem – activation, adoption, expansion, pricing, or organizational ownership – is important enough to justify attendance?
    Discovery, UX, and research capabilityACM CHI, UX360, UXLx, UXDX, UXPA International, uxcon, and Leading DesignDoes the team need a new method, stronger leadership practice, or better integration of research into product decisions?

    A balanced portfolio can contain different kinds of bets. An anchor event can provide a broad external scan and senior relationships. A specialist event can address a narrow operating problem. A virtual event can provide targeted content without travel. A regional event can deepen a market-specific network. You do not need every type; use only the ones connected to current priorities.

    This matters because superficially similar conferences can serve different jobs. A product-led growth event may be useful when the team is redesigning activation or commercial ownership. It is probably redundant when the same attendee recently explored the same questions and no resulting work has reached implementation. Repetition is valuable only when the audience, market, or decision has changed.

    AI-themed events deserve an especially strict filter. A prominent AI label does not tell you whether the agenda will help with real product work. Look for evidence that sessions address evaluation, data readiness, workflow adoption, reliability, governance, product economics, and organizational change – not only model demonstrations. If AI hiring is the objective, confirm that the likely participants include the builders or leaders you need to understand, rather than assuming a large technology audience will produce relevant candidate relationships.

    Keep some conference capacity uncommitted. Later announcements, agenda changes, and newly urgent company problems can make an event that looked optional more relevant than one selected during annual budgeting. A complete calendar created too early can leave no room for better information.

    Use calendar collisions to make sharper choices

    The 2026 schedule contains several useful collisions. They are not merely logistical problems. They reveal whether your selection criteria are strong enough to distinguish one event from another.

    On March 26 in New York, the Chief Product Officer Summit, Product Operations Summit, and Product-Led Summit occur on the same day, with the latter two continuing through March 27. One attendee cannot meaningfully cover all three. A product executive working through organizational design may favor the leadership audience. A product operations owner rebuilding planning and decision infrastructure has a different reason to be there. A growth leader should not choose either merely because colleagues are attending.

    Amsterdam creates a similar choice on May 28-29, when the Chief Product Officer Summit and Product-Led Summit run on the same dates. If both themes matter, split coverage only when each attendee has a distinct brief and a real route to implementation. Dividing the team without separate objectives simply doubles the expense and produces overlapping summaries.

    San Francisco offers a cluster rather than a direct collision: the Chief Product Officer Summit takes place on September 17, followed by Product-Led and Product Operations Summits on September 22-23. Combining them may reduce duplicated travel, but it also creates time away between events. The itinerary is justified only if the later event answers a separate, important question and the attendee can use the intervening time productively.

    Europe’s June schedule shows why geographic proximity can be misleading. Mind the Product runs in London on June 15-16, Growth Minded Superheroes is in Frankfurt on June 16, UX360 EU is in Berlin on June 23-24, Product-Led Summit is back in London on June 24-25, and Product at Heart follows in Hamburg on June 26. These cities can look like one efficient conference run on a map. In practice, conflicting dates, transfers, context switching, and prolonged absence can overwhelm the incremental learning.

    Use these rules when events overlap or cluster:

    1. For a same-city overlap, choose by problem. Split coverage only when the briefs are materially different and both attendees can act afterward.
    2. For a nearby-city cluster, calculate absence as well as travel. Add transfers, working days between events, and recovery time to the decision.
    3. For a cross-region collision, protect the portfolio. Select the event serving the more important company priority, not the one creating more fear of missing out.
    4. For a virtual alternative, distinguish content from access. Choose virtual participation when learning is the main objective; protect in-person travel for relationships or interactions that genuinely require presence.

    Also block the attendee’s return capacity before approving the trip. If the calendar is packed with internal meetings immediately after the event, synthesis and follow-up will be displaced by routine work. That turns an expensive learning opportunity into a folder of notes no one uses.

    Send the right attendee with an operating brief

    The most senior available person is not automatically the right attendee. Neither is the person most eager to travel. Match the attendee to the decision, the conversations required, and the authority needed afterward.

    Conference objectiveBest-positioned attendeeAuthority or support required afterward
    Reframe AI product strategyThe product or AI leader accountable for the portfolio decisionAbility to change priorities, commission validation work, or define a new investment thesis
    Improve product operationsThe leader or operator who owns planning, decision flow, tooling, or product ritualsA sponsor willing to change responsibilities, forums, or operating mechanisms
    Strengthen continuous discoveryA product, design, or engineering representative close to active customer and delivery workA real product area in which to apply the method and partners prepared to participate
    Calibrate executive leadershipThe product executive who owns the relevant organizational or stakeholder decisionAccess to the executive forum where the operating change will be decided
    Inform AI hiringThe person accountable for role design, assessment quality, or the hiring decisionPermission to update the role scorecard, sourcing thesis, or interview process

    A practitioner sent to an executive event may hear useful ideas but lack access to the people or forums needed to apply them. An executive sent to a method-heavy event may return with broad principles while missing the implementation detail. When the objective spans levels, assign a primary attendee and an internal sponsor instead of assuming one person can represent every perspective.

    The operating brief should travel with the attendee. Include:

    • Outcome statement: The decision or operating change this event should inform.
    • Question set: The questions that sessions and conversations must help answer.
    • Current hypothesis: What the team presently believes, including the conditions that might make it wrong.
    • Conversation map: Relevant peer operators, complementary functions, speakers, customers, partners, or prospective hires to look for.
    • Agenda rules: Sessions that directly answer the brief, useful alternatives, and content that can be skipped without regret.
    • Capture format: Claim, evidence, context, transferability, open question, and follow-up.
    • Coverage plan: Decisions delegated during the attendee’s absence and the escalation path for anything that cannot wait.
    • Return forum: The operating meeting where recommendations will be considered, not merely presented.

    Networking becomes easier when the attendee has a real problem to discuss. A useful introduction contains four things: your role, the situation you are working through, the precise question, and a modest request to compare approaches. That gives the other person something concrete to respond to. A generic request to connect transfers all the work to them.

    Do not optimize for the number of conversations. Optimize for relevance and continuity. A smaller set of exchanges tied to active work is more useful than a long contact list with no reason for another interaction. Before leaving the event, record why each follow-up matters and what question should move forward.

    Measure return through decisions and behavior

    Conference return is often reduced to attendance, notes, leads, or an internal presentation because those outputs are easy to count. None proves that the company made a better decision or changed how it operates.

    Use a decision ledger instead. Before the event, record the active decision, current hypothesis, unresolved evidence, owner, and forum where action can be authorized. After the event, update the same record with the strongest signal, its origin, the context in which it appeared to work, the resulting confidence change, and the next action.

    Evidence quality matters. A vendor claim, a conference-stage success story, a private conversation with an operator, and a pattern repeated across unrelated practitioners should not receive equal weight. The attendee should label which kind of signal they captured and identify what still needs internal validation. Conference learning can shape a test or decision; it should not bypass product judgment.

    Return channelEvidence that countsWeak proxy
    Decision qualityA documented product, AI, growth, or operating decision was changed, accelerated, narrowed, or deliberately held pending better evidenceA polished event summary
    ExecutionAn experiment, discovery activity, hiring change, or operating practice has an owner and a checkpointGeneral enthusiasm about trying new ideas
    NetworkA relevant relationship continues around an active problem, with a clear reason for follow-upContacts or badge scans collected
    TalentThe team improves a role thesis, evaluation approach, market map, or ongoing candidate relationshipA stack of resumes without fit assessment
    Knowledge transferColleagues can apply the insight to current work and understand the conditions under which it may failSlides placed in a shared folder

    Not every conference needs direct revenue attribution. Executive calibration, hiring insight, and discovery capability can create value indirectly. Forcing a speculative revenue number onto those outcomes creates false precision. It is still reasonable to demand a visible chain from attendance to evidence, from evidence to a decision, and from the decision to owned work.

    Cancel or reassign attendance when that chain is missing. A booked event should be reconsidered if the final agenda no longer matches the priority, the attendee cannot articulate a decision thesis, the same questions were recently explored elsewhere, or no one has capacity to act on the result. Sunk planning effort is not a reason to spend more time and money.

    Open your planning calendar with one real company priority, then review the published 2026 product conference dates. Shortlist the event whose audience and agenda best match the unresolved decision. Assign the decision owner, write the operating brief, protect the return forum, and only then approve the booking. If an event cannot survive that sequence, decline it and preserve the capacity for one that can.

    References

  • AI Product Management Skills: A Practical 12-Month Roadmap

    AI Product Management Skills: A Practical 12-Month Roadmap

    You may know how to prompt a model and still feel unprepared to own an AI product. That gap is real. Producing a plausible response is easy; deciding what should be built, how to evaluate it, when to trust it, and whether it improved the user journey requires a broader product skill set.

    The useful roadmap is not a queue of courses or tools. It is a sequence of increasingly consequential work: understand model behavior, turn ideas into testable artifacts, ship a bounded workflow, and then build the operating system that lets more teams do it responsibly.

    What you should be able to do after 12 months

    An AI product manager does not need to become a machine-learning engineer. You do need enough technical judgment to frame a feasible problem, challenge an architecture, inspect failures, define an evaluation, and make a release decision with engineering and design.

    The 12-month progression from foundations to governed scale works because each phase produces evidence needed by the next one. You learn model constraints before promising a user experience. You build evaluations before exposing the system to real customers. You prove one workflow before standardizing it across a product organization.

    Key takeaways

    • Months 1-3: Learn model behavior, context management, prompting, retrieval, privacy, and data governance. Apply them to product discovery.
    • Months 4-6: Build prototypes and an evaluation system. Instrument activation and retention before treating the feature as ready.
    • Months 7-9: Ship a bounded AI-enabled workflow with safeguards, monitoring, recovery paths, and clear human control.
    • Months 10-12: Standardize evaluation gates, analytics, discovery practices, roadmapping, and outcome-based reporting.

    Treat these as capability gates, not calendar milestones. If you cannot explain why a prototype failed in month six, more production infrastructure will not fix the problem. If you cannot show that users received value in month nine, scaling the feature will only distribute uncertainty.

    By the end of the roadmap, your portfolio should contain operating artifacts rather than course certificates: an AI product brief, a prompt and retrieval pattern, a reusable evaluation set, an instrumented production workflow, a risk checklist, and a scale playbook. Those artifacts demonstrate that you can move from possibility to accountable product performance.

    Months 1-3: Learn enough AI to make sound product decisions

    Your first objective is not technical fluency for its own sake. It is learning where model behavior changes a familiar product decision. A deterministic feature is expected to return the same result for the same state. A generative feature can produce different, incomplete, or confidently incorrect outputs. That changes acceptance criteria, testing, interface design, and the meaning of “done.”

    Build an operator’s mental model

    Work through four capabilities in order:

    1. Model behavior and constraints: Learn what the model receives, what it produces, where variability enters, and which failures matter to the user. You should be able to distinguish a capability problem from a context, instruction, or workflow problem.
    2. Context window management: Decide which information belongs in the model’s working context, which information is stale, and which information should never be sent. More context is not automatically better context. Irrelevant material can obscure the evidence the task actually requires.
    3. Prompting as product specification: Write reusable instructions that state the task, relevant context, constraints, required output, and quality criteria. Save the prompt with examples of both acceptable and unacceptable behavior. A prompt library is useful only when another person can reproduce and assess the result.
    4. Retrieval-first design: For tasks that depend on changing or proprietary knowledge, learn the basic pipeline: retrieve relevant approved information, give that information to the model, generate an answer, and preserve enough traceability to investigate failures. This is a product choice as much as an architecture choice because it determines what the experience can reliably know.

    Pair these capabilities with privacy-by-design and data governance from the beginning. Before using customer or company information, write down which data classes are permitted, who can access them, where they may be retained, and what must be removed or masked. If those answers are unclear, use synthetic or explicitly approved material until the policy is settled. Avoiding sensitive data at the prototype stage is safer than trying to remove it after it has spread through prompts, logs, and evaluation files.

    Apply the foundations to product discovery

    Discovery gives you a low-risk place to practise. Use generative AI to summarize research, cluster feedback, compare recurring needs, or sharpen a value proposition. Keep the model in an assistive role: every synthesized theme should remain traceable to the underlying customer evidence. If you cannot inspect the feedback behind a cluster, you cannot tell whether the model found a pattern or flattened important differences.

    Create an AI product brief for one candidate problem. Include:

    • The user and the job they are trying to complete.
    • The decision or work the model will assist with.
    • The inputs the system may use and the inputs it must reject.
    • The expected output and the conditions that make it useful.
    • The consequence of a wrong, missing, or delayed output.
    • The point at which a person reviews, edits, approves, or overrides the result.
    • The product signal that would show improved user behavior.

    You are ready for the next phase when you can explain the proposed experience without hiding behind model vocabulary. You should be able to identify the necessary context, name the important failure modes, explain whether retrieval is needed, and show how the user remains in control.

    Months 4-6: Prototype the experience and build its evaluation system

    A prototype is valuable when it tests uncertainty, not when it merely looks polished. Use generative AI to accelerate UX mocks, PRDs, in-app guidance, and alternative interaction flows, but spend the saved time on the questions that determine whether the product deserves to ship.

    Prototype the entire decision loop. Show where the user supplies context, how the result is presented, what happens when the answer is weak, how the user corrects it, and whether that correction improves the next step. The error state is part of the primary AI experience; hiding it until engineering integration creates false confidence.

    Use evaluation as a development method

    Eval-driven development turns a vague judgment such as “the answers seem good” into a repeatable product decision. Build the evaluation alongside the prototype:

    1. Define the task boundary. State what the system is expected to do and what remains outside its responsibility.
    2. Collect representative cases. Include normal inputs, ambiguous inputs, missing information, adversarial behavior, and cases where the correct response is to stop or ask for clarification.
    3. Write a scoring rubric. Assess the properties the user actually needs, such as correctness, relevance, completeness, appropriate tone, traceability, or compliance with a constraint.
    4. Record a baseline. Compare the proposed experience with the current workflow or a simpler non-AI alternative. A model output is not valuable merely because it exists.
    5. Inspect failure patterns. Separate prompt failures, missing-context failures, retrieval failures, model limitations, interface confusion, and policy violations. Each category points to a different remedy.
    6. Set a release gate. Decide which failures block launch, which require human review, and which are tolerable in the intended use case. The gate should reflect the consequence of error, not enthusiasm for the feature.

    Keep the evaluation set versioned with the product. When you change the prompt, model, retrieval logic, or available tools, rerun the same cases. Otherwise, an apparent improvement in one example can conceal regressions elsewhere.

    Instrument behavior before launch

    Quality evaluation and product analytics answer different questions. An evaluation tells you whether the system behaved acceptably on known cases. Behavioral analytics tells you whether customers reached value in the product.

    Define the journey in Amplitude or your existing analytics system before exposing the prototype broadly. Capture the moment a user encounters the feature, supplies enough information, receives an output, accepts or edits it, completes the downstream task, returns to use it again, abandons it, or escalates to a person. That sequence gives you activation and retention signals rather than a vanity count of generations.

    If you run an A/B test, choose the minimum detectable effect before launch. The decision matters because an experiment that cannot detect a product-relevant change may produce an inconclusive result even when the dashboard looks busy. Define the primary outcome, guardrail metrics, exposure rule, and analysis plan before looking at the results.

    Move forward when the prototype solves a defined task, the evaluation catches meaningful failures, the events expose the user journey, and the experiment can answer a decision. A persuasive demo without those four elements is still a demo.

    Months 7-9: Ship a bounded workflow, not an open-ended assistant

    The production phase is where product judgment becomes visible. Start with a workflow that has a recognizable beginning, end, and owner. Customer-support, CRM, and guided-onboarding workflows are useful patterns because the AI can sit inside an existing user journey rather than asking customers to invent a use case from a blank chat box.

    Screen the workflow before committing engineering capacity:

    • Is the user’s job clear enough to define a successful completion?
    • Does the system have access to approved, relevant context?
    • Can you observe whether the user accepted, corrected, ignored, or escalated the output?
    • What happens to the customer if the system is wrong?
    • Can a consequential action be paused, reviewed, or reversed?
    • Is a generative system materially better than a rule, search result, template, or conventional workflow?

    Use agentic AI only when the job genuinely requires several connected steps, tool use, or changing plans. Additional autonomy also creates more places for permissions, context, and actions to go wrong. Begin with the narrowest useful boundary, then expand it when production evidence supports the change.

    Map the production loop before building it

    A product trio should be able to trace the complete workflow on one page:

    1. Trigger: What user action or system event begins the workflow?
    2. Context: Which profile, conversation, account, or knowledge records are retrieved?
    3. Generation or decision: What does the model produce, classify, recommend, or plan?
    4. Tool action: Which systems can it read from or write to, and under whose authority?
    5. Human checkpoint: Which output can be edited, rejected, or approved before it changes customer data or sends an external message?
    6. Recovery: How does the product handle low confidence, missing data, tool failure, timeouts, or a user correction?
    7. Learning signal: Which feedback updates the evaluation set, product decision, or workflow design?

    Place safeguards at the point of consequence. Restrict the data and tools the workflow can access. Require explicit approval before a high-impact external action. Preserve a record of the inputs, retrieved context, output, action, and user response so a failure can be investigated. If an action cannot be safely reversed, keep it behind human review until the risk has been addressed.

    Threat detection and response also need a product playbook. Define what counts as suspicious input or abnormal behavior, who receives the alert, how the workflow is disabled or contained, what evidence is retained, and how affected users are handled. The escalation path should exist before the first serious incident, not be improvised during it.

    Monitor the experience at four levels

    • User outcome: Did the customer complete the intended job with less effort or fewer avoidable handoffs?
    • AI quality: Are the evaluation scores and failure categories changing after releases?
    • Workflow health: Are retrieval, model, and tool steps completing as expected, and can the team locate the failing stage?
    • Risk: Are users overriding outputs, escalating cases, encountering policy violations, or triggering suspicious behavior?

    Track deployment frequency because a team that can release safely can also learn faster. Do not confuse release frequency with customer value, though. The useful loop connects a deployment to a quality change, a behavior change, and a decision about what to do next.

    Months 10-12: Turn one successful product into a repeatable system

    Scaling is not copying the same AI feature into every surface. It is making the successful practices reusable while preserving room for different user risks and workflow requirements.

    Codify the operating assets that reduced uncertainty during the earlier phases:

    • An intake template that starts with the user problem, current workflow, expected outcome, and consequence of error.
    • A continuous-discovery practice that keeps generated themes connected to original customer evidence.
    • A retrieval-first architecture template for products that depend on approved or changing knowledge.
    • A shared prompt library with owners, versions, expected behavior, and known limitations.
    • An evaluation gate covering representative cases, blocking failures, human-review requirements, and regression checks.
    • A production checklist covering permissions, privacy, observability, recovery, threat response, and user control.
    • A monitoring cadence that connects product behavior, AI quality, workflow health, and risk.

    Do not impose one universal quality threshold on every AI feature. A low-consequence drafting aid and a workflow that changes a customer account do not carry the same downside. Use the same evaluation process across teams, but set release gates according to the task, affected user, reversibility, and consequence of failure.

    Use common analytics without erasing product context

    A unified analytics model lets leadership compare lift across products without forcing every team to use an identical funnel. Standardize the basic meanings of exposure, meaningful use, successful task completion, correction, abandonment, escalation, and return usage. Then let each product define the events that represent those states in its own journey.

    This is also where roadmapping and sprint planning should move from output commitments to outcome-based decisions. “Ship an AI assistant” is an output. A useful objective describes the customer behavior or business result that should change. The roadmap can then contain competing ways to produce that change, including improvements that do not require AI.

    Use a consistent stakeholder narrative:

    • What shipped: The workflow or capability placed in users’ hands.
    • What moved: The user, product, quality, and risk signals that changed.
    • What was learned: The assumptions confirmed, rejected, or still unresolved.
    • What happens next: The decision to expand, revise, contain, or stop the work.

    That structure prevents activity from masquerading as progress. It also gives executives a clear basis for funding decisions: evidence of value, evidence of control, and a specific next bet.

    Start this week with one recurring user decision. Write its AI product brief, run the workflow manually with permitted data, and save the successful and failed cases as the beginning of an evaluation set. If you cannot define a good result or the consequence of a bad one, stay in discovery. If you can, you have a concrete first artifact and a reason to proceed to a prototype.

    References

  • Analytics-Led Product Growth: A Practical Operating System

    Analytics-Led Product Growth: A Practical Operating System

    Your dashboards are busy, the roadmap is full, and every team can produce a chart that supports its preferred priority. Yet when activation changes or retention weakens, nobody can say with confidence which customer behavior moved, why it moved, or what decision should follow.

    That is the problem analytics-led product growth should solve. It connects a customer outcome to an observable behavior, a trustworthy measurement, and a product decision. Build that chain well and analytics becomes part of how you choose, test, and scale growth bets – not a reporting layer added after the roadmap is set.

    Start with the decision, not the dashboard

    A useful metric has a job. It helps you make a defined decision about a defined customer journey. If nobody can explain what would change when the metric rises, falls, or stays flat, the metric is decoration.

    Before asking an analyst to build a chart, write the decision you are trying to make. Use this sequence:

    1. Name the business outcome. Examples include durable revenue, lower cost-to-serve, or greater adoption of a valuable workflow.
    2. Name the customer outcome that must occur first. A customer may need to complete setup, receive an approval, publish something, invite a collaborator, or finish another meaningful job.
    3. Identify the observable behavior that proves the customer reached that outcome. A login or button click rarely proves value on its own.
    4. Choose the leading metric that will reveal movement soon enough to guide a decision.
    5. Add guardrails for consequences you are unwilling to trade away, such as errors, support contacts, failed verification, or degraded retention.
    6. State the decision in advance: if the primary metric moves and the guardrails remain healthy, what will you ship, stop, expand, or investigate?

    This creates a small driver tree. At the top is the result the business needs. Under it are the customer behaviors capable of producing that result. Beneath those sit the product changes you can test. It keeps the team from mistaking a feature launch for progress.

    For example, “launch a new onboarding tour” is an output. “Increase the share of eligible new customers who complete onboarding and reach first value, without increasing support contacts” is an outcome. The second formulation tells you what to measure, which trade-off to protect, and how to judge the work. That is why connecting a north star, outcome-based objectives, and trustworthy instrumentation matters before experimentation begins.

    Be equally precise about activation. Activation is not whatever event produces the most convenient chart. It is the earliest behavior that credibly indicates the customer has experienced meaningful value. You should be able to explain why that behavior matters and verify whether customers who complete it behave differently later. A relationship with retention is evidence worth investigating, but it is not proof of causation.

    Instrument the journey so the numbers can be trusted

    Growth analysis breaks when the event model describes the interface instead of the customer journey. “Button clicked” tells you that an interaction happened. “Application submitted successfully” tells you that the customer completed a meaningful step. Instrument the confirmed outcome whenever the product can observe it.

    A usable event taxonomy needs more than consistent names. For each critical event, document:

    • The exact behavior represented by the event.
    • The condition that causes it to fire, including whether it records an attempt or a confirmed success.
    • The properties needed for legitimate analysis, such as customer profile, plan, entry channel, product surface, or journey variant.
    • The identity rule that connects anonymous activity, authenticated users, and accounts.
    • The event owner and the product change that introduced or modified it.
    • Known exclusions, delayed events, retries, and duplicate-event behavior.

    The distinction between attempt and success is especially important. If an event fires when a customer selects “Submit,” it can overstate completion when validation, verification, payment, or a server error prevents the operation from finishing. Record the attempt when it helps diagnose friction, but use the confirmed success event to measure conversion.

    Test the instrumentation by completing the journey yourself in a controlled environment. Confirm that events appear once, in the expected order, with the expected identity and permitted properties. Then test an error path, a retry, an interrupted session, and a return on another session. A tidy taxonomy document cannot compensate for events that fire inconsistently in the product.

    Data quality also needs an operating guardrail. Watch for sudden volume changes, missing properties, impossible event sequences, duplicate events, and identity merges that shift historical counts. Assign an owner to investigate those conditions. Otherwise, a tracking defect can enter a roadmap discussion disguised as a customer trend.

    In regulated or trust-sensitive journeys, collect only the properties needed for an approved purpose. Do not place sensitive customer values in event names or unrestricted properties. Verification steps, rejection reasons, and error details can be analytically useful, but careless collection can create privacy, access-control, and regulatory exposure. Apply privacy-by-design and data-governance rules before the event reaches the analytics platform, not after it has been copied into dashboards.

    This foundation is not analytics housekeeping. A precise event taxonomy with explicit data-quality guardrails determines whether activation, retention, and experiment results are credible enough to guide investment.

    Read growth as a sequence of customer behaviors

    No single metric can explain growth. Read the journey as a sequence: the customer arrives with intent, reaches first value, returns for value, adopts more of the useful workflow, and does so without creating unsustainable service costs. Each stage answers a different question.

    Journey stageUseful signalsQuestion to answerDiagnostic cuts
    First valueActivation rate, onboarding completion, time-to-first-valueAre eligible new customers reaching the first meaningful outcome, and how much effort does it require?Customer profile, entry channel, plan, journey version
    ConversionStep conversion and end-to-end funnel conversionWhere does demonstrated intent fail to become a completed outcome?Error state, verification path, device or product surface
    RetentionD7, D30, and D90 cohort retentionWhich customers return at a meaningful interval after starting or activating?Start cohort, activation behavior, customer profile
    DepthFeature adoption and weekly-to-monthly active ratioIs recurring value broad and repeated, or concentrated in shallow activity?Key workflow, account maturity, role or use case
    Service economicsSupport contact rate and cost-to-serveIs growth creating a scalable customer experience?Journey step, error category, customer profile

    D7, D30, and D90 are observation points, not universal definitions of healthy retention. Choose intervals that match the product’s natural usage cycle and state the qualifying behavior. “Returned” could mean opening the product, completing the core workflow, or receiving recurring value. Those definitions produce different answers.

    Cohorts protect you from another common mistake: combining customers who began at different times. Group customers by a meaningful start event and period, then compare like with like. If a change affected only new customers, an all-user average can hide its impact. If one customer profile improved while another declined, the aggregate can falsely imply stability.

    Start diagnosis at the narrowest point where behavior diverges. If onboarding completion falls, inspect the step-level funnel and error states before redesigning the whole experience. If activation rises but D30 retention does not, test whether the activation definition captures real value or merely easier completion. If adoption grows alongside support contacts, inspect whether customers are discovering value or being forced through confusing work.

    Benchmarks help you calibrate the baseline and find unusually weak stages, especially when you can compare activation, time-to-first-value, funnel conversion, retention, adoption, and cost-to-serve. They are not targets to copy blindly. Confirm that the compared products use compatible populations, events, intervals, and definitions. Then use the gap to choose where to investigate, not to declare the solution.

    Turn a behavioral signal into a disciplined experiment

    An unusual funnel drop or cohort difference is a clue. It becomes a product bet only after you identify a plausible mechanism. Move from observation to hypothesis with one sentence: for a defined segment, changing a defined part of the experience should change a defined behavior because of a stated customer problem.

    Every experiment brief should contain:

    1. The observed behavior and the segment in which it occurs.
    2. The customer problem or mechanism that could explain it.
    3. The proposed change and the behavior it is intended to influence.
    4. One primary outcome metric tied to the hypothesis.
    5. Guardrail metrics covering important downstream or risk consequences.
    6. The minimum detectable effect, or the smallest difference that would be meaningful enough to change the decision.
    7. The allocation, eligibility rules, analysis window, and stopping rule defined before results are inspected.
    8. The action attached to each possible result: ship, revise, stop, investigate, or collect more evidence.

    The minimum detectable effect helps determine whether an A/B test can answer a decision responsibly. Setting it after seeing the data defeats its purpose. If the effect you care about requires more eligible traffic or time than the decision can support, narrow the question, choose a larger meaningful change, or use discovery evidence to reduce uncertainty. Do not label an underpowered result as proof that the idea works or does not work.

    Not every problem deserves an A/B test. Fix a confirmed tracking defect before interpreting the metric. Fix a severe error or harmful experience when withholding the repair would be irresponsible. Use an experiment when there is genuine uncertainty about how a product change will affect behavior and a valid comparison can resolve that uncertainty.

    Read outcomes without spin. If the primary metric improves and the guardrails remain acceptable, the change has earned consideration for rollout. If the primary metric is flat, do not rescue the result with an unrelated secondary metric chosen afterward. If a guardrail deteriorates, investigate the trade-off even when the primary metric wins. If the result is inconclusive, record it as inconclusive and decide whether the remaining uncertainty justifies more investment.

    In-app guidance is a good example of why the outcome matters more than the intervention. Guide views, tooltip clicks, and tour completion describe exposure. The actual question is whether the intended customer reaches value sooner, completes the journey, adopts the useful feature, or needs less assistance. A stack that combines product analytics, in-app guidance, segmentation, and controlled testing can connect those layers, but the tools cannot choose the right success definition for you.

    Build an operating cadence that changes the roadmap

    Analytics-led growth becomes real when a metric review ends in an owned decision. A recurring meeting that only narrates charts creates reporting work, not product learning. Separate journey diagnosis from portfolio allocation so each conversation has a clear purpose.

    Run a weekly journey review

    Use the weekly review to inspect one or two critical journeys, not every dashboard. Product, design, engineering, and the relevant data partner should arrive with the same metric definitions. Add risk, operations, support, or go-to-market partners when the journey crosses their responsibilities.

    • Begin with the customer outcome and the eligible population.
    • Review movement in the primary metric, guardrails, and important segments.
    • Separate data-quality issues from actual behavior changes.
    • Identify the earliest journey step where the affected cohort diverges.
    • Choose one decision: repair instrumentation, fix a known defect, continue discovery, launch a test, expand a result, or stop work.
    • Record the owner, next evidence, and decision date.

    A short decision log is valuable because it preserves what the team believed before the result was known. Record the observation, hypothesis, metric definition, decision, and eventual outcome. This prevents old ideas from returning without new evidence and makes changes to metric definitions visible.

    Use a monthly portfolio review for allocation

    The portfolio review should decide where product capacity goes. Compare opportunities using the size of the affected segment, the severity of the broken journey, the connection to a strategic outcome, the strength of the evidence, the cost of learning, and the downside represented by guardrails. This is where benchmarks, discovery, experiment results, and commercial context meet.

    Require every material roadmap bet to identify its driver metric and measurement plan. An initiative can still be strategically necessary when immediate experimental proof is unavailable, but that uncertainty should be explicit. Do not disguise a conviction bet as a data-backed conclusion.

    Keep objectives focused on outcomes rather than delivery. A roadmap item may be a redesigned verification flow, a product tour, or a new workflow. The objective should describe the customer and business result. The key results should show whether the relevant behavior improved while guardrails remained healthy. That structure gives product, risk, operations, and go-to-market partners a common basis for trade-offs.

    Key takeaways

    • Begin with a product decision and customer outcome; build the dashboard only after both are clear.
    • Define activation as evidence of first value, not as signup, login, or another convenient activity event.
    • Instrument confirmed outcomes, attempts, and error states separately so conversion can be diagnosed accurately.
    • Read activation, conversion, retention, adoption depth, and service economics as a connected journey.
    • Use cohorts and meaningful segments before trusting an aggregate trend.
    • Define the primary metric, guardrails, minimum detectable effect, and stopping rule before running an A/B test.
    • End every analytics review with a decision, an owner, and the next evidence required.

    Choose one growth journey this week. Write down the first valuable customer outcome, audit the events needed to reconstruct it, and identify the one decision your current data should support. That small exercise will show you whether analytics is guiding the product or merely describing it.

    References

  • 2026 Support Capacity Playbook: Bold AI Automation, Smarter Staffing, Zero‑Surprise SLAs

    Capacity planning has always been a high-stakes exercise in customer service, and when you miss, the signal shows up fast in backlogs and SLAs. I’ve lived that pressure across multiple cycles, and 2026 will reward teams that plan differently. AI fundamentally changes capacity planning because it changes the work. It resolves the bulk of your volume, speeds up execution, and elevates the complexity and value of what humans handle. The consequence is simple: planning models must evolve. This is the final installment in my 2026 customer service planning series, and I’m focusing on the tension every leader feels right now—be ambitious about automation, but avoid the trap of understaffing if your assumptions don’t hold. My goal is to share how AI changes the logic of capacity planning, what I’ve learned implementing these practices with my team and with customers, and the common traps to avoid. Traditional planning rests on relatively stable assumptions: volume grows predictably, work types stay consistent, handle times don’t swing dramatically, and productivity improves slowly with better tools and training. In an AI-first model, none of that is guaranteed, and the fundamentals flip. The mix of work changes as AI absorbs a growing share of simpler conversations, leaving humans with deeper, more time-consuming issues that demand human-to-human connection. Demand can actually increase when you remove friction, so AI can both resolve more and attract more volume. Human time splits differently as teammates solve customer problems and also review AI behavior, give feedback, improve content, and support system-level work. Performance becomes dynamic, not fixed—automation rate isn’t a one-time number; it can rise with care and fall with neglect. If you plan for 2026 using a pre-AI model—assuming similar productivity, similar work mix, and a linear relationship between volume and headcount—you will underestimate what it now takes to run a high-performing support organization. There are many metrics you can track, but the one to put at the center is automation rate (AI Agent involvement rate × AI Agent resolution rate). This single construct tells me what share of total volume AI actually resolves, how much work remains for humans, how much additional demand humans can absorb, and how ambitious I can be with headcount. Early in the journey, I prioritize raising involvement—getting the AI involved in more conversations. Once involvement is high, I shift to resolution on the hardest remaining work, where each additional 1% of automation can represent several people’s worth of capacity. In my 2026 plans, automation rate sits alongside projected inbound volume, average “output” per person for the more complex work that remains, and occupancy—how much time is allocated to customer-facing interactions versus operational and strategic work. Together, those inputs give a realistic picture of how many people you need and where they should spend their time. First, plan boldly on automation, but match it with investment. I do not cap automation assumptions at 40–50% “because AI is new.” Many teams are already modeling 60%, 70%, even 80%+ for 2026—when they invest in AI ownership and content. The investment is non-negotiable: named ownership for AI performance (AI ops, knowledge management, conversation design), clear automation targets by work type (e.g., informational vs. personalized vs. actions vs. deep troubleshooting), realistic expectations for what’s easy to automate and what’s not, and a concrete plan to raise automation over time in monthly or quarterly steps rather than a single jump. To decide where to invest first, I dig into the data. I start with the biggest volume drivers, separate content-led issues from those dependent on data or complex procedures, assume higher resolution potential for content-led topics once the knowledge base is in shape, and set more modest initial resolution expectations for system-dependent flows. Then I stair-step improvements as the systems, data contracts, and workflows mature. In short, bold automation goals only work when paired with the team structure, content, and systems required to reach them—and the discipline to iterate. Second, expect human “output” per person to go down. That’s a mindset shift. Historically, we assumed individual productivity would stay flat or tick up as tools improved. In an AI-first model, humans handle fewer conversations but more complex, cross-functional issues—and create more value despite lower case counts. I model a lower “cases closed per person” than prior-year baselines, explicitly assume the remaining work is more complex and time-consuming, and redefine productivity to include system-level work like AI Agent improvements, content updates, and policy or workflow change management. I also report “capacity created” from automation alongside human outputs, so leadership sees the full picture. Third, rethink occupancy: more time off the queues, on higher-value work. Traditional occupancy splits time between inbox and training, meetings, and breaks. Now there’s an expanding “out-of-inbox” portfolio that directly affects AI performance and overall capacity: reviewing AI-handled conversations, improving AI Agent triaging and handovers, contributing to content and procedures, feeding insights to product and engineering, and supporting system changes that reduce future volume. I set lower inbox occupancy targets than before and make the rationale explicit. People aren’t working less—they’re working differently. In planning, I assume more time spent on improvement and system work, make it visible (for example, X% in inbox and Y% on AI and system improvement), and treat this as critical, not a “nice to have.” If you don’t proactively allocate it, it won’t happen—and your automation and performance targets will suffer. Fourth, work with the finance team early, and treat your plan as a set of assumptions. Capacity planning with AI is a set of bets across automation rate, human output, demand growth, occupancy, and where surplus capacity (if any) goes. I bring finance in early, show that the plan is dynamic and directly tied to AI performance, and label every lever as an assumption with ranges. I commit to a quarterly review cadence with finance to compare assumptions versus reality and adjust headcount, targets, and investment as needed. The risks are real: if automation grows slower than expected and you stop backfilling too early, you’ll be understaffed for months. Hiring and onboarding take time, so course-correcting late creates strain. If you do produce surplus capacity, have a clear strategy to reallocate those teammates to higher-value work—improving systems, feeding insights back to product, supporting new channels, and driving proactive CX—rather than defaulting to reductions. I also set explicit guardrails—if automation rate misses by five points for two consecutive months, we pause planned reductions and revisit hiring gates. If it over-performs, we shift people into backlog eradication, content upgrades, or proactive outreach, so we bank compounding value. To set your team up for success in 2026, anchor your plan on automation rate, be honest that humans will handle fewer but harder conversations, and protect time for system improvements. Partner early and often with finance, avoid shrinking too fast, and design a plan for surplus capacity so you’re never caught flat-footed. If AI is going to handle the majority of your customer conversations, your plan has to be designed to help it do that well and to keep your team set up for meaningful, sustainable work. A 2026 plan built on adaptable assumptions—not fixed predictions—will hold up as your work, your systems, and your customers’ expectations continue to change. If you’d like future editions like this, subscribe and stay close—I’ll keep sharing what’s working, what isn’t, and how to tune your customer support AI strategy in real time.

    Inspired by this post on The Intercom Blog.

    Book a consult png image
  • Year-End Reflection for Product Leaders: Values, Themes, and the 100‑Wishes Reset

    Year-End Reflection for Product Leaders: Values, Themes, and the 100‑Wishes Reset

    I’ve been closing the year with a deliberate reflection ritual for more than a decade, and this season I found fresh energy for it after listening to an insightful conversation with Teresa Torres and Petra Wille on All Things Product. Their approaches mirror the evolution many product leaders experience: moving from rigid annual goal-setting to values-led themes, longer time horizons, and a healthier respect for spaciousness. In my own practice, that shift has created better focus, less pressure, and far more meaningful outcomes.

    Prefer to listen? You can find this episode here: Spotify | Apple Podcasts. I took notes with my team in mind and translated the discussion into a simple, values-driven framework that any product organization can adopt.

    Why does annual reflection matter for product people? Because our work lives at the intersection of ambiguity, trade-offs, and time. If we only measure ourselves by shipped output or quarterly OKRs, we overlook the compounding value of learning, relationships, and judgement. I treat this ritual as a strategic reset: a chance to surface patterns, adjust expectations, and recommit to outcomes over output.

    My own reflection habit started scrappy—paper notebooks, messy timelines, and even artful visualizations inspired by Dear Data by Giorgia Lupi & Stefanie Posavec. Like Petra, I’ve found that tactile, analog artifacts unlock insights I miss in a spreadsheet. Over time, I’ve kept the spirit and simplified the mechanics: a “what went well” review, a short list of hard lessons, and a handful of decisions that paid off—or didn’t.

    The biggest evolution for me has been moving from rigid annual goals to values and themes. I still run OKRs, but I use them to track progress, not identity. The lens of process vs. outcome goals—reinforced by ideas from Atomic Habits—helped me set fewer, better commitments. For example, instead of “launch X by Y,” I’ll emphasize the cadence of customer discovery, the health of the product trio, and the quality of decisions made along the way.

    One exercise that changed my practice is the “100 wishes” list. It’s powerful—and surprisingly difficult. Pushing past 30 or 40 wishes forces me to name latent interests and long-range intentions I rarely say out loud. Combined with decade-level themes, the list helps me balance ambition with patience. I don’t try to do it all next year; I use it to spotlight direction, not deadlines.

    I also review patterns across years: Where did over-scheduling create hidden costs? When did I protect focus time and what did that unlock? Paul Graham’s Maker’s Schedule, Manager’s Schedule remains a useful calibration tool here. And when I feel the pull toward constant throughput, I revisit Stefan Sagmeister’s The Power of Time Off (TED Talk) to remind myself why strategically creating space often yields the most valuable ideas.

    Of course, not every year follows plan—and that’s normal. Reflection helps me spot unrealistic expectations early and let them go. When setbacks hit, I’ll rewatch Dealing with Setbacks and re-ground in continuous discovery. The question isn’t “Did we do everything?” but “Did we learn fast, protect customer value, and make trade-offs aligned with our values?” That’s how empowered product teams compound impact.

    My sharing philosophy has become more nuanced over time. Some reflections are public to invite dialogue and accountability; others stay private so I can process honestly. I’ve found it helpful to publish what I’m saying no to, capture a theme for the year ahead, and keep the rest for myself and my team. This balance preserves motivation while still contributing to the broader product management leadership community.

    If you’re designing your own ritual, consider this lightweight flow: review wins and tough calls, write your “100 wishes,” extract a few values-based themes, then translate those into process goals for Q1. Revisit monthly, not just annually. If you like structured prompts, Chris Guillebeau’s How to Conduct Your Own Annual Review from The Art of Nonconformity offers a practical template you can adapt to your context.

    For deeper dives and complementary ideas, I bookmarked these as part of my year-end reset: What I’m Saying No to This Year—And Why, Ask Teresa: My Leaders Still Want Roadmaps with Timelines—What Should I Do?, Scaling Impact: A Look at the Year Ahead (2022), Let’s Connect in 2025: A Look at the Year Ahead, The Interview Coach, and Petra’s own year-ahead reflections (here and her 2026 version). I also recommend revisiting the prior conversation on leadership and change: Role of Leadership in Transformations.

    I’d love to hear how you approach your end-of-year reflection. What questions bring you the most clarity? Which practices help you set an intentional, values-driven path for the next year? Share your process—I’m always looking to learn from other product creators and leaders.

    Inspired by this post on Product Talk.

    Book a consult png image
  • Inside the Engine Room: How I Drive Scalable Analytics APIs, Reliability, and Performance

    Inside the Engine Room: How I Drive Scalable Analytics APIs, Reliability, and Performance

    I build and scale analytics platforms with a product mindset, and the work starts with the "middleware and compute systems that power analytics at scale." In platforms like Amplitude analytics and other unified analytics platform architectures, that foundation is what makes everything else possible.

    Day to day, I oversee the "APIs behind charts, cohorts, and metrics—driving performance, reliability, and platform scalability." When those APIs are fast and resilient, every product team—from growth to customer success—can trust the insights they use to ship, learn, and iterate.

    From an engineering leadership standpoint, I partner closely with SRE to define SLOs and error budgets, wire CI/CD pipelines for safe deploys, and track DORA metrics so we improve speed without compromising quality. This combination reduces incident management toil and shortens MTTR while keeping data freshness and query latency within strict thresholds.

    From a product management leadership lens, the goal is clarity: crisp APIs, predictable contracts, and transparent stakeholder management across data, engineering, and GTM teams. That alignment empowers product teams with reliable cohorts and metrics, accelerates experimentation, and de-risks roadmaps.

    If you’re scaling analytics, invest first in the platform layer: middleware and compute, schema governance, caching strategies, and cost-aware compute. Do that well, and the visible experience—charts, cohorts, and metrics—feels effortless, even as you grow to serve billions of events with confidence.

    Inspired by this post on Amplitude – Best Practices.

    Book a consult png image
  • A Practical Measurement System for B2B Product-Led Growth

    A Practical Measurement System for B2B Product-Led Growth

    Your dashboard can show more sign-ups, more activated users, and more feature adoption while the business becomes no healthier. In B2B, that usually happens when measurement stops at individual activity and never proves that an account reached repeatable value, stayed engaged, or developed credible expansion potential.

    You don’t need a larger metric catalog. You need a connected measurement system that follows one account from eligibility to first value, repeated value, retention, and commercial impact. That system should also tell your team where the journey broke and which decision to make next.

    Measure one customer journey at two levels

    B2B products create value through people, but the commercial relationship usually exists at the account, organization, or workspace level. This creates a measurement problem: user metrics and account metrics can each look healthy while hiding a different weakness.

    Growing active-user counts may only mean that existing customers added more seats. Growing active-account counts can conceal dependence on one enthusiastic user inside each account. Measure both levels, but don’t blend them into an ambiguous active-customer number.

    If your billing or value unit isn’t an account, substitute the correct economic entity, such as a workspace or billable organization. The important rule is that every metric names the entity being counted.

    Before building a dashboard, write a metric contract for every top-line measure. It should specify:

    • The business question and decision the metric supports.
    • The entity being counted: user, account, workspace, or revenue.
    • The qualifying population and the moment an entity becomes eligible.
    • The event or event sequence that constitutes success.
    • The observation window and the period allowed for success.
    • Exclusions for employee activity, test accounts, duplicate identities, and unusable telemetry.
    • The segments that must remain available for diagnosis.
    • The owner responsible for resolving definition or data-quality problems.

    This contract prevents a common denominator error. Invited members may create new user registrations, but they aren’t necessarily new accounts. If they enter the activation denominator as though they started a new buying journey, the rate stops answering a coherent question.

    Your event model must also resolve each action to the account or workspace in which it occurred. Assigning an event to a user’s current account can corrupt historical reporting when that user belongs to multiple workspaces or changes organizations.

    Decision questionPrimary unitUseful measuresWhat a weakness helps you locate
    Did a new account reach meaningful value?Account or workspaceActivation rate and time-to-first-valueAcquisition quality, setup friction, or an unclear value path
    Is value becoming repeatable?Account and user roleRecurrence of the core behavior, active accounts, and feature adoptionShallow adoption, novelty effects, or dependence on one champion
    Does usage endure?Account cohortCohort-based product retentionA gap between initial success and durable value
    Is product value creating commercial pull?Account and revenueValidated expansion intent plus expansion and contraction revenueA weak commercial signal, packaging mismatch, or failed handoff
    Can the experience scale responsibly?Account and operationsSupport deflection, incident signals, and delivery guardrailsGrowth that is shifting cost or reliability problems elsewhere

    A useful portfolio view therefore combines activation, onboarding completion, time-to-first-value, active accounts, feature adoption, cohort retention, expansion and contraction revenue, and support deflection. These aren’t interchangeable scorecard tiles. Each one answers a different question in the value chain.

    Treat activation as a hypothesis about future retention

    Activation isn’t whatever happens at the end of your onboarding checklist. It is your current hypothesis about the earliest observable behavior that shows a qualified account has received meaningful product value.

    That distinction matters. In a hypothetical collaboration product, inviting a colleague may be necessary setup. Completing a shared workflow may be the first evidence of value. Calling the invitation activation would reward the team for moving people through configuration, even if the product never solves the underlying job.

    A credible activation definition should meet several tests:

    • It represents delivered value, not mere exposure to a screen or feature.
    • It occurs early enough for product, marketing, and customer-success teams to influence it.
    • It can be measured consistently for the eligible population.
    • It respects different use cases when those use cases have materially different value paths.
    • It is associated with stronger later retention inside comparable cohorts and segments.

    The last test is important, but it doesn’t establish causality. Accounts that activate may already have greater intent, better internal sponsorship, or a more suitable use case. Treat the relationship as evidence that improves your hypothesis, then use controlled interventions where practical to learn whether removing a particular barrier changes downstream behavior.

    Use the same contract to define the related measures. Activation rate is the share of eligible accounts completing the activation behavior within the agreed window. Time-to-first-value begins at the same eligibility moment and ends at the same success event. Onboarding completion remains a diagnostic measure unless completing onboarding itself delivers the promised outcome.

    A practical validation loop looks like this:

    1. Map the path from eligibility through setup to the proposed first-value event.
    2. Use funnels and segmentation to locate the step where qualified accounts stop progressing.
    3. Compare later retention for accounts that did and didn’t complete the candidate behavior within equivalent use-case, acquisition, and account cohorts.
    4. Inspect the time-to-first-value distribution by segment instead of relying on one blended average.
    5. Test a focused intervention at the identified bottleneck, such as simpler setup, clearer messaging, a contextual guide, or a revised product tour.
    6. After any activation lift, check repeated use and cohort retention before declaring that the growth system improved.

    This is where funnels, high-signal behavioral segments, retention cohorts, and A/B tests on messaging or in-app guidance belong in the same workflow. The funnel identifies friction. The cohort tests whether the behavior matters. The experiment tests whether your intervention changes it.

    If activation rises while later retention stays flat, don’t celebrate the dashboard. Either the activation behavior is too shallow, the experiment generated temporary compliance, or the product fails to deliver enough value after the first success. Each explanation produces a different roadmap decision.

    Use a driver tree to show exactly where growth breaks

    A flat scorecard tells you what changed. A driver tree shows where to investigate. For many B2B PLG products, the measurement chain can be expressed as:

    Eligible accounts → setup complete → activated → repeated core value → retained active accounts → expansion intent → expansion or contraction revenue.

    This isn’t a universal linear funnel. Renewal and expansion can overlap with ongoing adoption, and different roles may enter at different points. Its purpose is to expose the assumptions connecting product behavior to business performance.

    Read movement between adjacent stages before reaching for a broad explanation:

    • If eligible accounts grow while activation falls, split acquisition quality from product friction. Compare equivalent acquisition and use-case segments before changing onboarding.
    • If onboarding completion improves while activation doesn’t, you probably removed checklist friction without improving the first-value experience.
    • If activation improves while repeated value doesn’t, inspect whether the activation event is too shallow or the initial experience creates novelty rather than a durable habit.
    • If active users increase while active accounts remain flat, adoption may be deepening inside existing customers without broadening the account base.
    • If repeated product value is healthy while account retention or revenue weakens, product telemetry alone can’t explain the result. Join account behavior with customer status and commercial data.
    • If expansion-intent signals rise while expansion revenue stays flat, validate the signal and inspect the go-to-market handoff before assuming the product created qualified demand.

    These patterns narrow the search; they don’t prove a cause. A driver tree should help your team decide which segment, journey step, qualitative evidence, or experiment to inspect next.

    The same tree separates leading indicators from lagging outcomes. Setup completion and high-signal power-user actions can lead into active usage and cohort retention, while expansion and contraction revenue arrive later. A leading metric earns its place only when you continue testing its relationship with the outcome it is supposed to predict.

    This changes how you write product OKRs. “Launch a new onboarding tour” is an output. “Increase validated activation for qualified accounts without weakening downstream retention or support outcomes” is an outcome. The first statement rewards shipping. The second forces the team to state the behavior it expects to change and the evidence required to keep investing.

    For every experiment, record the target segment, affected driver, hypothesis, exposure event, primary outcome, guardrails, analysis window, and downstream validation. Don’t call a variant successful because it increased tutorial clicks when the intended outcome was account activation.

    Keep operational guardrails beside growth outcomes. Incident management and DORA measures can complement product metrics when faster experimentation or adoption adds reliability risk. Support deflection provides another check: apparent growth is less attractive if it merely transfers unresolved friction to customer support.

    Segment for decisions, then use benchmarks for calibration

    A blended retention curve is an average of customers who may have different jobs, expectations, acquisition paths, and product cadences. It can improve because your customer mix changed even when no segment received a better experience.

    Build cohorts from a consistent starting event, such as the moment an account becomes eligible to pursue first value. Then define retention using a value-bearing behavior appropriate to the product’s natural cadence. A product used for an occasional but critical workflow shouldn’t be forced into a weekly-use definition merely because weekly activity is easy to chart.

    Keep three concepts separate:

    • User retention asks whether a person or role continues using the product.
    • Product-level account retention asks whether the original account cohort continues completing the qualifying value behavior.
    • Commercial retention asks what happened to the cohort’s revenue after expansion and contraction.

    One cannot substitute for another. An account may retain its contract while meaningful product use declines. Another may show healthy usage while commercial contraction occurs. That gap is information, not an inconvenience to smooth out.

    Start with segments that can change an actual decision:

    • Primary use case or job-to-be-done, when value paths differ.
    • Self-serve versus sales-assisted acquisition, when expectations or onboarding support differ.
    • Account size or plan, when collaboration depth and feature access differ.
    • Administrator, champion, and end-user roles, when each role contributes differently to value.
    • New versus established accounts, when the same behavior means something different at each lifecycle stage.

    Resist slicing until every cell becomes noisy. A useful test is simple: if a segment underperforms, would you choose a different intervention or owner? If not, it probably doesn’t belong on the operating dashboard.

    Treat expansion intent with the same discipline as activation. Seat invitations, adoption of a higher-value workflow, or repeated encounters with a product limit may be plausible candidates, but none should be accepted on intuition alone. Compare each signal with later expansion outcomes by account segment. Keep the label “intent” until the behavior proves commercially predictive.

    Sales involvement doesn’t invalidate product-led measurement. Keep a shared lifecycle definition, segment the acquisition or expansion motion, and distinguish product-sourced, product-assisted, and merely product-active accounts using explicit attribution rules. Otherwise, any active customer can be retroactively called product-led.

    External benchmarks are most useful after your internal definitions are stable. Before comparing rates, verify the unit of analysis, eligibility rule, event semantics, observation window, segment mix, and treatment of assisted accounts. Peer-informed targets can calibrate ambition and help identify gaps, but a benchmark built from a different denominator is not a target. It is a false comparison.

    Your operating view should ultimately answer four questions without a forensic exercise: Which segment moved? At which stage? Did a downstream outcome confirm the movement? What decision changes because of it? If a metric can’t help answer one of those questions, it belongs in a diagnostic workspace rather than the executive scorecard.

    B2B PLG measurement FAQ

    Should the headline metric count users or accounts?

    Use the economic value unit for the headline and user-level measures for diagnosis. For most B2B products, that means retained active accounts or workspaces completing a validated value behavior. Role-based user measures then reveal whether adoption is broad, concentrated in a champion, or blocked for a critical participant.

    What is the best north-star metric for B2B product-led growth?

    There is no context-free north-star metric. Choose a value-bearing account behavior that naturally recurs and has a defensible relationship with retention. Pair it with activation, expansion and contraction, and reliability guardrails so one optimized number cannot hide damage elsewhere.

    Should sales-assisted accounts be excluded?

    No. Excluding them can remove a material part of the customer journey and overstate the independence of the product motion. Keep the lifecycle and value definitions consistent, label the acquisition or expansion path, and compare segments. Product-led growth doesn’t require sales-free growth; it requires clarity about what product behavior contributed.

    When should the activation definition change?

    Change it when the product’s value proposition, target job, telemetry, or evidence linking activation with retention materially changes. Version the definition and avoid splicing incompatible measures into one time series. Backfill the new definition only when the historical event data supports it; otherwise, mark a clean break.

    Before your next roadmap review, write the metric contracts for one activation behavior and one retained-account behavior. Connect them to expansion and contraction, add a reliability or support guardrail, and ask every major roadmap bet to name the link it should move. If a bet can’t state its expected behavioral outcome and downstream confirmation, you have found a strategy gap before spending the engineering effort.

    References

  • Long-Horizon Company Building: How to Operate for Decades

    Long-Horizon Company Building: How to Operate for Decades

    You are looking at a roadmap full of credible near-term work, yet none of it seems likely to change your company’s position. The team is busy, customers are asking for improvements, and every investment has a reasonable explanation. What is missing is a clear connection between today’s choices and the company you want to become.

    Long-horizon company building solves that problem only when it changes how you allocate capital, sequence capabilities, learn from customers, and stop work. A 25-year ambition is not permission to wait longer for results. It is a decision filter that helps you distinguish compounding investments from activity that merely fills the next planning cycle.

    Choose a problem that becomes more defensible with time

    Not every company should play a decades-long game. Time does not rescue weak demand, an undifferentiated product, or a market whose underlying problem is disappearing. A long horizon is useful when the work required to serve customers creates assets that become more valuable as they accumulate.

    Before you commit to a long-horizon strategy, test the problem against a few concrete conditions:

    • The pain is structural. Customers are constrained by an enduring workflow, infrastructure dependency, procurement model, or service failure. The opportunity does not depend entirely on a temporary technology cycle.
    • Frustration and switching costs are both high. Switching costs alone protect incumbents. Frustration alone can produce shallow demand for a convenient feature. When customers are dissatisfied but cannot change easily, a substantially better end-to-end experience can open a durable market.
    • The solution requires cumulative capability. Reliability knowledge, operational data, installation expertise, distribution, hardware, service operations, or customer trust should improve with continued use. If a new entrant can reproduce your advantage quickly, waiting longer will not make the business stronger.
    • The first product creates credible adjacencies. Expansion should follow the same customer, capability base, or service promise. A list of unrelated markets is not a platform strategy.
    • The customer outcome can support the business model. The way you charge should reinforce the result customers buy, rather than reward complexity they would prefer to avoid.

    The sharpest test is simple: explain why the company should be structurally better after years of serving customers. Your answer must identify a mechanism. More telemetry may improve diagnosis. More deployments may reduce installation risk. Deeper workflow integration may increase the value of adjacent services. Trust may lower the friction of adopting the next product. Merely having more customers or more features is not enough.

    A useful thesis takes this shape: for a specific customer, a costly problem will persist because of a structural constraint; repeatedly building a named capability will improve a defensible advantage; controlling certain interfaces is necessary to deliver the promise; and observable evidence will tell you when the thesis is weakening.

    If you cannot complete that logic without relying on market size, ambition, or executive conviction, you do not yet have a long-horizon strategy. You have a long-range hope.

    Convert a 25-year belief into present-day decisions

    A decades-long horizon should not produce a decades-long roadmap. The farther out you look, the less credible feature-level precision becomes. Preserve the direction while making the route explicitly revisable.

    Separate your strategy into three layers:

    • Enduring commitments: the customer you serve, the problem you believe will remain important, the experience you intend to make possible, and the principles you will not trade away casually.
    • Revisable hypotheses: the product architecture, distribution motion, ownership boundary, pricing model, and capability sequence that currently appear most likely to deliver the promise.
    • Disposable work: features, prototypes, internal systems, campaigns, and implementation choices. These deserve no protection beyond the evidence they produce.

    This separation prevents two common errors. The first is strategic thrashing: changing the destination whenever a current bet disappoints. The second is strategic stubbornness: defending a failed implementation because it has been wrapped in the language of mission.

    Meter provides a useful example of the distinction. The company maintained its commitment to a full-stack networking service while spending more than four years in early research and development. It also discarded about a year of operating-system work. The durable thesis survived; a costly implementation did not. That is what conviction looks like when it remains accountable to learning.

    At each planning cycle, require every major initiative to answer four questions: Which lasting capability will this build? What customer evidence should it produce? What finding would cause you to reshape or stop it? What are you deliberately declining so the investment receives enough attention?

    The stop condition matters most. Without one, patient capital quietly becomes protected capital. Teams learn to explain delays instead of testing assumptions. Write the condition while enthusiasm is high, before sunk costs and personal identity enter the decision.

    Key takeaways

    • Use a long horizon to define durable commitments, not detailed forecasts.
    • Fund work that compounds a named capability or reduces a consequential uncertainty.
    • Protect the customer problem and company promise, not the current implementation.
    • Give every major bet observable evidence and an explicit stop condition.
    • Treat abandoned work as a valid strategic outcome when it prevents a larger misallocation.

    Own only the stack required to keep the promise

    Vertical integration is neither inherently bold nor inherently wasteful. It is justified when a layer you do not control repeatedly prevents you from delivering the outcome customers believe they purchased.

    Start with the promise, not the architecture. Map the complete path from customer intent to customer outcome:

    • How the customer evaluates and buys the product
    • How the product is installed, configured, and activated
    • Which interfaces determine performance and reliability
    • What telemetry reveals failure before or after the customer notices
    • How support diagnoses and resolves a problem
    • Which service commitment makes the outcome commercially credible

    Mark every point where an external dependency can break the promise. Then ask whether tighter integration would materially improve the experience and whether the capability will compound across customers or future products. Own a layer when both answers are strong. Keep partnering when the dependency is replaceable, the layer is genuinely commodity-like, or internal ownership would add cost without improving the customer outcome.

    This prevents full-stack ambition from turning into organizational vanity. Building hardware, software, installation operations, support tooling, and service delivery at once creates many ways to fail. The burden of proof belongs with the added ownership. Each new layer should remove a specific failure mode, improve a measurable part of the promise, or unlock a strategically important product that would otherwise remain impossible.

    Physical-product teams should also treat geography as part of the operating design. When design, manufacturing, and iteration depend on one another, physical proximity can compress feedback loops. Meter used Shenzhen in this way during its development. The general lesson is not that every hardware company needs the same location. It is that organizational geography should follow the bottleneck: put the people making interdependent decisions close enough to learn at the speed the product requires.

    The business model belongs in the same analysis. If customers want an outcome but must assemble vendors, equipment, installation, and support themselves, packaging the complete experience as a service can reduce complexity and clarify accountability. Service commitments then become part of the product, not language added after the product is built. The company earns recurring revenue by continuing to deliver the outcome, which aligns incentives more closely than a transaction that ends when equipment changes hands.

    Distribution should reinforce learning during the early stages. A direct sales motion gives product and commercial leaders access to the buyer’s language, objections, procurement constraints, implementation concerns, and definition of value. That access is especially important when you are trying to establish seller-market fit: the ability to identify the right buyer, explain the value consistently, navigate the buying process, and deliver what was sold.

    Before adding channel distance, verify that target buyers recognize the same problem, objections fall into understandable patterns, sales commitments survive the implementation handoff, and the economics support the promised service. A channel can scale a repeatable motion. It cannot repair one that the company does not yet understand.

    Replace planning theater with a customer-learning system

    Removing OKRs does not create focus. It removes one alignment mechanism. If you do not replace it with a visible decision system, priorities will depend on executive proximity, persuasive storytelling, and whichever escalation arrived most recently.

    A lightweight operating system still needs a few explicit artifacts:

    • A strategic narrative: the customer problem, the long-horizon thesis, the current constraint, and the choices the company is making because of them.
    • A primary customer-value measure: evidence that the promised outcome is actually occurring, not merely that work shipped.
    • Guardrails: reliability, service, economics, or trust conditions that must not deteriorate while the primary outcome improves.
    • An unhappy-customer ledger: a shared record of broken promises, stuck use cases, escalations, and gaps between what was sold and what was delivered.
    • A decision log: the assumption behind each consequential choice, the evidence available at the time, the owner, and the condition for revisiting it.

    The unhappy-customer ledger is often more useful than another aggregate dashboard. A satisfaction score compresses many experiences into one number. An escalation exposes the precise boundary where your product, service, sales process, or ownership model failed.

    For every serious case, capture the customer’s intended outcome, the point at which progress stopped, the expectation that was violated, the immediate resolution, and the systemic change required. Classify that change as product, operations, sales, support, or ownership-boundary work. Then look for recurring failure modes across cases.

    Do not let this become a larger support queue. Closing the individual ticket is necessary, but the strategic value comes from removing the class of failure. If customers repeatedly struggle during installation, the answer may be a better workflow, different telemetry, a narrower promise, or ownership of an interface that has been treated as someone else’s problem.

    This system also clarifies empowerment. A product team should know the outcome it owns, the constraints it must respect, the decisions it can make independently, and the conditions that require escalation. Empowerment without a clear outcome produces local optimization. Authority without proximity to customer evidence produces slow, brittle decisions.

    The same clarity applies to performance problems. A company cannot preserve a long horizon while allowing unresolved role or behavior gaps to consume the team’s attention. Define the gap, the expected standard, the support available, the decision owner, and the process for reaching a fair conclusion. Move quickly toward clarity, while still following the appropriate people process. Delayed ambiguity is not patience.

    Make patience accountable in your next strategy review

    Long-horizon work will contain periods when visible output understates real progress. Research, infrastructure, reliability, manufacturing, and operational design may need to mature before customers see the complete benefit. The leadership challenge is to distinguish that legitimate incubation from drift.

    Patience is working when the core customer thesis remains supported, important uncertainties are being resolved, a reusable capability is getting stronger, and customer failures are becoming better understood or less frequent. The dates may move, but the quality of evidence improves.

    Drift looks different. Milestones move without producing new knowledge. Teams defend work by describing its difficulty or the effort already invested. The same customer failures return without a systemic response. Adjacent products receive attention before the original promise is dependable. Leadership keeps adding resources because it has not defined what would justify stopping.

    Review the portfolio by decision, not by project status. Continue work that compounds a necessary capability. Reshape work when the thesis remains sound but the current method is failing. Stop work whose original assumption no longer holds. Keep adjacent opportunities separate until the core business has earned the capacity to pursue them.

    You can run the review with the following sequence:

    1. Write the customer promise in language a buyer would recognize.
    2. Name the structural reason the problem should remain worth solving.
    3. Identify the capability that should become more valuable as the company learns.
    4. Map the interfaces, operations, and commercial dependencies that can break the promise.
    5. Examine recent unhappy-customer cases for repeated failure modes.
    6. For every major investment, write the evidence expected and the condition that would cause a change of course.
    7. Remove work that neither improves the current promise nor builds a required future capability.
    8. Assign the next consequential decision to a named owner with access to the relevant customer evidence.

    Do not leave that review with a more elaborate long-range deck. Leave with fewer bets, clearer ownership, explicit learning goals, and at least one piece of work you are prepared to stop.

    At your next planning meeting, ask which current investment will make the company structurally better at solving its chosen problem. If nobody can name the capability, the evidence, and the customer promise it serves, pause the work before time turns activity into strategy by accident.

    References

    • Shivam.Consulting Blog — Playing the 25-Year Game: Rethinking Networking, Ditching OKRs, and Owning the Full Stack
  • Operationalizing AI: A Practical System for Scalable Growth

    Operationalizing AI: A Practical System for Scalable Growth

    Your AI pilot works in the demo. Then it reaches a live workflow and slows down: the data is incomplete, nobody owns the exceptions, reviewers apply different standards, and the team cannot prove whether the result improved revenue, cost, speed, or retention.

    The gap is not model quality alone. Scalable growth requires an operating system around the model: a constrained business outcome, a mapped workflow, approved data, explicit decision rights, measurable quality, controlled releases, and a path for handling failure. Build those pieces around one valuable use case, and AI can become a repeatable business capability instead of a collection of pilots.

    Choose the growth constraint before the AI use case

    Do not begin with a broad instruction to “find an AI use case.” That framing encourages teams to start with a model capability and search for somewhere to place it. Start with a constrained business problem instead.

    The unit of investment should be a decision or task inside a customer or employee journey. “Build a churn copilot” is too broad. “Before a renewal review, summarize approved usage and CRM signals, identify the evidence of risk, and propose an action for the customer success manager to review” is narrow enough to test.

    Most growth-oriented opportunities fit into four useful lanes:

    • Revenue: improve qualification, conversion, expansion, cross-sell, or win-back decisions. Measure the commercial event, not the number of AI recommendations generated.
    • Efficiency: reduce the cost, handling time, rework, or backlog associated with a repetitive process. Good candidates have high task volume and outputs that can be checked without recreating the work.
    • Speed: shorten a discovery, delivery, or release cycle. If the workflow serves software delivery, deployment frequency can be relevant, but it is not evidence of customer or commercial value by itself.
    • Activation and retention: make onboarding, guidance, or support more contextual. Measure whether customers reach the intended product behavior and continue receiving value, not whether they clicked an AI-generated tooltip.

    A disciplined portfolio can pair one revenue use case with one efficiency use case, define success before development, and release each through a narrow MVP. That balance matters. An efficiency-only roadmap can shrink costs without creating differentiation, while an unconstrained revenue bet can consume attention without proving economic value.

    Screen each candidate with the same questions:

    • What business metric should move, and what is its current baseline?
    • Which person, decision, and moment in the workflow create that movement?
    • Does the task occur often enough to justify a reusable solution?
    • Are the required inputs available, current, and approved for this purpose?
    • Can a reviewer distinguish an acceptable result from an unacceptable one?
    • What happens when the system is wrong, and can the action be reversed?
    • Who owns the outcome after the launch team moves on?

    My test is blunt: if you cannot name the workflow event, the owner, the baseline, and the failure consequence, you do not yet have an implementation candidate. You have a discovery question. Fund the learning needed to answer it before funding scale.

    Convert the use case into a controlled workflow

    An AI feature becomes operational when its behavior is defined inside the surrounding work. That means understanding what happens before the model is called, what the model may do, how its output is checked, and what happens next.

    Begin by mapping the task as it is performed, choosing one step to augment, selecting the right automation method, and iterating against an explicit quality bar. Do the task manually while mapping it if the real process is unclear. Policy documents often describe the intended path; observation reveals the exceptions that determine whether automation will survive production.

    1. Name the trigger. Specify the event that starts the workflow, such as a support request, renewal review, onboarding milestone, invoice submission, or product release.
    2. Identify the inputs. Record each system, document, field, permission, and freshness requirement. Separate required evidence from optional context.
    3. Expose the decisions. Write down the classifications, judgments, calculations, and approvals a person currently makes. Hidden judgment is where apparently simple automations tend to break.
    4. Specify the output. Define its schema, audience, channel, timing, and acceptable evidence. “Produce a helpful answer” is not a specification.
    5. Map exceptions. Include missing records, contradictory inputs, unsupported requests, low-confidence cases, policy conflicts, and unavailable downstream systems.
    6. Assign each step to code, retrieval, an LLM, or a person. The workflow should use the simplest reliable mechanism for each job.
    7. Define the handoff. State who reviews the result, what they can change, when the workflow must stop, and where failures are recorded.

    Use each form of automation for the work it can control

    Use deterministic code for exact calculations, validation rules, permissions, routing, and other behavior that should produce the same answer from the same inputs. Use an LLM where language is ambiguous, inputs are unstructured, or the task requires drafting, summarizing, extracting, or classifying meaning.

    When the answer must reflect company facts, policy, or customer history, retrieve the approved information at runtime instead of expecting the model to remember it. A retrieval-first design can connect behavioral and CRM context to account signals and recommended actions, while preserving a visible trail back to the evidence used.

    Keep a person in the path when the consequence is material, the action is difficult to reverse, or the definition of a correct result remains contested. Human review is not a permanent excuse for weak quality, however. The reviewer needs defined criteria, enough context to make a decision, and an easy way to correct and categorize the failure.

    Write an execution contract, not just a prompt

    A production instruction set should define more than tone and role. Treat it as an execution contract containing:

    • the objective and the business context;
    • the permitted inputs and authoritative evidence;
    • the decision criteria the system must apply;
    • the required output structure;
    • the actions it may and may not take;
    • the conditions that require refusal or escalation;
    • the way uncertainty should be represented;
    • examples of acceptable, unacceptable, and edge-case behavior.

    For an agentic workflow, increase authority in deliberate stages: observe, draft, recommend, act after approval, and only then act within defined limits. Do not jump from a convincing chat demonstration to autonomous execution. Agentic AI needs explicit guardrails and verifiable quality before it can safely take work out of a human queue.

    Measure business value, workflow performance, and AI quality separately

    A dashboard that reports requests, tokens, or generated answers tells you that the feature was used. It does not tell you whether the business improved. You need separate measures because an AI system can look healthy at one layer while failing at another.

    Measurement layerWhat to trackWhat it reveals
    Business outcomeConversion, expansion, cost per completed outcome, cycle time, activation, or retentionWhether the investment affects the growth constraint it was chosen to address
    Workflow performanceCompletion, rework, exception, escalation, abandonment, and end-to-end latencyWhether the surrounding process can absorb and use the AI output
    AI qualityCorrectness, evidence support, instruction adherence, output validity, and appropriate refusalWhether the system behaves acceptably across expected and difficult cases
    Risk and operationsUnauthorized data exposure, prohibited actions, overrides, incidents, rollback events, and unresolved failuresWhether growth is being purchased with unacceptable operational or trust costs

    Build the measurement path before the rollout:

    1. Capture the baseline. Measure the existing workflow using the same outcome definition you will use after launch. Otherwise, a faster AI step can hide slower review, higher rework, or shifted labor elsewhere.
    2. Create a representative evaluation set. Use permitted examples from normal, difficult, and failure-prone cases. Define the expected result and the critical errors for each case.
    3. Weight failures by consequence. Formatting errors, unsupported factual claims, privacy failures, and unauthorized actions should not disappear into one average score.
    4. Run offline evaluations before exposure. Test the complete combination of instructions, model, retrieval, tools, and output validation. A model score alone does not represent the production system.
    5. Release behind a feature flag. Start with a controlled cohort, preserve the ability to roll back, and compare outcomes. Use A/B testing when assignment and outcome measurement are credible; use a phased rollout when they are not.
    6. Record versions. Log the model, instructions, retrieval configuration, tools, and policy version associated with each result so a regression can be traced.
    7. Turn failures into future tests. Categorize meaningful production failures and add them to the evaluation set before the next release.

    This is the practical meaning of eval-driven development: instrument the system, watch for drift, and tighten the delivery loop while changes remain controlled by feature flags. It turns evaluation from a launch checkpoint into part of product development.

    Use a scale gate that includes economics

    Do not scale because the demo is impressive or employees like the interface. Require four decisions:

    • The business outcome is moving in the intended direction, or there is credible evidence that the workflow is producing the leading behavior tied to it.
    • Quality remains acceptable across normal cases, edge cases, and high-consequence failures.
    • Total cost per successful outcome is viable after model usage, retrieval, storage, human review, escalation, rework, and operations are included.
    • The operating owner can detect, contain, and learn from failures without depending on the original project team.

    If a pilot fails one of these gates, the decision is not automatically to cancel it. Narrow the scope, change the workflow, improve the evidence, or stop. What matters is that expansion is earned by measured behavior rather than assumed from adoption.

    Scale through guardrails, reusable components, and clear ownership

    Governance should make routine decisions faster. When every team has to rediscover which data is permitted, which evaluation is sufficient, and who can approve a release, governance becomes a sequence of meetings. When those expectations are encoded in a standard launch record, teams know the path before they build.

    Create a minimum launch record for every workflow

    • the business outcome, baseline, and accountable owner;
    • the workflow boundary, users, and authorized actions;
    • the approved data sources, access controls, retention rules, and prohibited data;
    • the evaluation set, acceptance criteria, and critical failure classes;
    • the human review and escalation conditions;
    • the logging, monitoring, feature flag, and rollback plan;
    • the model, retrieval, tool, and vendor dependencies;
    • the incident owner and the method for notifying affected internal teams or customers when appropriate.

    Privacy-by-design, data governance, red-teaming, and defined review gates are growth infrastructure. They reduce repeated risk debates and make the safe path reusable across launches.

    If a workflow touches personal data, confidential customer content, employment decisions, payments, security actions, or contractual commitments, involve the appropriate privacy, security, legal, financial, or people owner before live use. The downside is not limited to a poor answer. The workflow can expose restricted data or take an action the business cannot easily reverse.

    Assign ownership beyond launch

    Four responsibilities must be explicit, even when one person holds more than one:

    • Business outcome ownership: decides whether the workflow is worth continuing based on the target metric and economics.
    • Workflow ownership: manages exceptions, reviewer behavior, process changes, and user feedback.
    • Technical ownership: controls releases, versions, integrations, reliability, monitoring, and rollback.
    • Risk ownership: defines the policy boundary and approves material changes to data, authority, or exposure.

    This prevents a common operating failure: the product team treats launch as completion, while the operations team inherits a changing probabilistic system without the tools or authority to manage it.

    Standardize the recurring parts, not every local process

    Once working use cases expose recurring needs, turn those needs into shared capabilities. Useful candidates include identity and permissions, governed retrieval connectors, evaluation tooling, instruction and model versioning, observability, feature flags, rollback controls, and cost attribution.

    Keep the final workflow close to the business team that understands the customer, exceptions, and outcome. Centralize the controls and infrastructure that should be consistent. This creates leverage without forcing every function into the same process.

    Review the portfolio as a set of products, not permanent projects. The decision for each workflow should be to expand it, fix a known constraint, narrow its authority, or retire it. Continuous discovery with product trios can refine the prompts, data sources, and experience while evidence determines what scales and what stops.

    Operationalizing AI: three questions leaders ask

    Should you build a central AI platform first?

    Usually, no. Start with the minimum secure infrastructure required for a valuable workflow. Standardize a component when several use cases need the same capability or when inconsistency creates material risk. Data access, identity, logging, and release controls may need early consistency; a broad internal platform without proven workflows can become an expensive set of assumptions.

    How do you know a pilot is ready to scale?

    A pilot is ready when it improves the intended business or workflow outcome, stays within quality and risk boundaries, has viable cost per successful outcome, and can be operated without daily intervention from its builders. Usage and positive comments are supporting signals, not a scale decision.

    Where should a human remain in the loop?

    Keep human approval where consequences are high, actions are difficult to reverse, evidence is incomplete, or acceptable judgment cannot yet be specified. Remove or reduce review only when evaluations and production monitoring show that the remaining risk is understood and controlled. A reviewer who merely clicks approve without adding judgment is not a guardrail; it is latency disguised as governance.

    For your next AI proposal, require a one-page charter containing the outcome, workflow boundary, owner, baseline, approved data, evaluation set, failure policy, release plan, and full cost model. If a line is blank, fund discovery to resolve it. If the charter is complete, release the smallest useful workflow behind a control, learn from real failures, and widen its authority only when the evidence earns it.

    References

  • How to Build a Self-Improving AI Support Operation

    How to Build a Self-Improving AI Support Operation

    Your AI support agent handled the easy questions, produced an encouraging early lift, and then stopped getting better. The same topics still reach human agents. Content fixes happen when someone remembers. The aggregate resolution rate moves, but nobody can explain why.

    If that describes your operating review, a newer model is unlikely to be the first thing you need. You need a closed operating loop: every weak conversation becomes evidence, every useful insight gets an owner, and every change is tested against the next conversation it is meant to improve.

    Measure the improvement loop, not just resolution rate

    A self-improving support operation is not an agent that quietly rewrites or retrains itself. It is a managed system in which live conversations expose failure modes, people convert those failures into controlled changes, and later conversations show whether the changes worked.

    Resolution rate is an outcome of that system, not a diagnosis. An aggregate rate cannot tell you which intent deteriorated, why the agent handed a customer to a human, or whether a change repaired one topic while damaging another. It can also be misleading when eligibility changes. Expanding automation into harder intents may lower the rate while increasing the number of conversations resolved. Excluding difficult intents can produce the opposite effect.

    Start by documenting exactly what your denominator includes and what counts as a resolution. Keep that definition stable enough to compare periods, and report resolved volume alongside the rate. Then add the views that turn a dashboard into a work queue:

    • Coverage: Which inbound conversations are eligible for AI handling, and which are excluded?
    • Outcome by intent: Where does the agent resolve, hand off, or fail to answer?
    • Failure reason: Was the problem missing knowledge, weak retrieval, incorrect behavior, poor routing, or an issue the product itself must solve?
    • Quality: Did an audit, repeated contact, reopened conversation, or another trusted signal indicate that the apparent resolution was weak?
    • Change throughput: How many identified failures are waiting for diagnosis, testing, approval, or release?

    The intent-level view matters because it gives the owner somewhere to act. A falling aggregate rate is merely a warning. A cluster of unresolved questions about one feature, tied to one failure reason, is a tractable product and operations problem.

    Classify the failure before choosing the fix

    Teams waste cycles when every poor answer is treated as a documentation problem. Use a small failure taxonomy to route each issue to the layer that can actually repair it.

    Failure classWhat you observeLikely action
    Knowledge gapNo current, approved answer existsCreate or repair the canonical content
    Retrieval gapThe answer exists, but the agent does not receive or select itImprove structure, segmentation, metadata, or retrieval configuration
    Behavior gapThe right information is available, but the response is incomplete or misappliedAdjust instructions, examples, or agent configuration
    Routing gapThe agent should escalate but does not, or the handoff loses essential contextChange escalation conditions and the handoff payload
    Product gapNo support answer can resolve the underlying problemSend the evidence to product or engineering instead of disguising it as a content task

    This distinction prevents two common errors: endlessly rewriting accurate content when retrieval is broken, and asking the support agent to explain around a product defect that requires an actual fix.

    Give one owner the authority and the improvement queue

    Shared participation is useful. Shared accountability is not. One person should own the performance of the AI support operation, even though support, product, content, engineering, and security may contribute to individual changes.

    The title can be AI operations lead, support operations specialist, or something else. The mandate is what matters: identify underperforming intents, maintain the improvement backlog, coordinate changes across functions, enforce the evaluation process, and report what improved or regressed.

    Ownership becomes especially important after the launch surge fades. At Dotdigital, performance held at about 2,800 resolved conversations per month for three consecutive months. The response was to create a dedicated support operations specialist role focused on snippets, content, and the agent’s resolution capability. The lesson is not that every company needs the same job title. It is that a plateau without an empowered owner tends to remain a plateau.

    Do not bury improvement work in the general support queue. A customer ticket can close while the underlying failure remains. Create a separate, persistent record for the system-level issue, with fields that make it possible to trace evidence through to an outcome:

    • Representative conversation links and the affected intent
    • The observed failure and its customer consequence
    • The failure class and the evidence supporting that diagnosis
    • The knowledge, retrieval, behavior, routing, or product artifact to change
    • The accountable owner and required reviewer
    • The evaluation cases that must pass
    • The release status, version, and deployment date
    • The live signal that will be checked after release

    Define done as more than content published or configuration changed. An improvement is complete only when the change is linked to its originating evidence, reviewed at the appropriate risk level, tested, released, and checked in live operation.

    For prioritization, assess recurrence, consequence, confidence in the diagnosis, and effort separately. Do not let raw volume make the decision by itself. A rare failure involving access, privacy, or an irreversible customer action can deserve attention before a frequent wording problem. Conversely, a recurring low-risk knowledge gap may be the best candidate for a fast content repair.

    Turn live failures into governed, testable changes

    Feedback does not improve an agent merely because it was collected. A thumbs-down, a handoff, or an unresolved conversation is a signal, not a root cause. The operating loop has to convert that signal into a specific hypothesis and then close the loop.

    1. Collect: Group common handoffs and unresolved conversations by intent instead of reading them as isolated tickets.
    2. Diagnose: Assign a failure class and confirm that the proposed layer is actually responsible.
    3. Prioritize: Select the issue using recurrence, consequence, confidence, and effort.
    4. Change: Modify the smallest responsible artifact rather than making broad agent changes by default.
    5. Evaluate: Test the originating failures, realistic variations, and already-passing cases that could regress.
    6. Release and observe: Record what shipped, monitor the affected live intent, and feed any new failure back into the queue.

    Write the hypothesis before making the change: for this intent, changing this artifact should reduce this failure reason without degrading these existing behaviors. That sentence forces clarity about what success means and which regression cases belong in the evaluation set.

    When a live failure reveals a missing case, promote it into the regression set after the fix. Over time, the evaluation suite becomes a practical memory of mistakes the operation should not repeat. That is where compounding comes from: the team is not merely correcting answers; it is preserving each correction as a reusable control.

    Match governance to the blast radius

    Fast iteration and responsible review are compatible when the rules are explicit. A useful governance model distinguishes changes by consequence:

    • Low blast radius: A correction to an approved fact, an obsolete product step, or a missing limitation can follow a lightweight peer review and the relevant evaluation cases.
    • Moderate blast radius: Retrieval, behavior, and routing changes that can affect several intents should receive cross-functional review and a controlled release.
    • High blast radius: Actions involving permissions, account access, customer data, money, or security need stronger approval, a safe test environment, a rollback path, and an obvious route to a human.

    A wrong explanation can create confusion. A wrong action can change an account or expose data. Treating those changes as equivalent either slows harmless content repairs or makes consequential automation unsafe.

    Use focused sprints without making improvement episodic

    A concentrated sprint is useful when the backlog has accumulated or a set of topics is visibly underperforming. In one focused Anthropic effort, the team audited unresolved queries, repaired weak content, converted recurring macros into AI-usable snippets, and monitored live performance. That is a practical pattern for clearing known gaps quickly.

    The sprint should strengthen the standing loop, not replace it. Keep the same taxonomy, backlog, review rules, and evaluation artifacts after the concentrated work ends. Otherwise, the operation improves during special events and drifts between them.

    Make the improvement work visible in each operating review. Show the failure observed, the artifact changed, the evaluation result, and the live outcome or next check. Name the person who drove the repair. This rewards the behavior that creates durable gains instead of celebrating only a headline rate that few people can explain.

    Make AI-ready knowledge part of product launch readiness

    Company-specific support knowledge does not appear because the underlying model is capable. The agent needs current, approved information in a form it can retrieve and apply. Missing or contradictory knowledge is an operating failure, not a model mystery.

    Treat knowledge as production infrastructure. Every topic needs an owner. Important changes need versions and effective dates. Retired instructions need to be removed or clearly superseded. The agent’s ingestion and retrieval path needs verification, just as the customer-facing help experience does.

    A canonical source of truth does not have to be one enormous help article. It means there is one approved origin for the product facts from which help-center content, agent snippets, human macros, and other downstream formats are derived. When those formats are authored independently, contradictions are almost inevitable.

    Add an AI support gate to the new product introduction process. Before a feature is considered ready, confirm that:

    • A named owner is accountable for keeping the feature’s knowledge current.
    • The canonical material explains what changed, who can use it, how it works, and where its boundaries are.
    • Known limitations and escalation conditions are explicit rather than left for the agent to infer.
    • The effective version or release state is clear, so old and new instructions cannot be confused.
    • The content has been ingested or indexed and retrieval has been tested.
    • Expected support intents and representative evaluation cases are ready before inbound volume arrives.
    • Support has a defined path for returning launch-day failures to product, engineering, or the knowledge owner.

    This is not only administrative hygiene. In my organization, embedding a canonical source of truth into launch readiness has consistently supported resolution rates above 50% for new features from day one. That result is evidence for the operating model, not a universal benchmark; intent mix, product complexity, and the definition of resolution still matter.

    Do not automatically turn every human answer into permanent knowledge. First decide whether the resolution is generalizable. If it is, update the canonical material. If it is a legitimate exception, encode the escalation path. If the underlying issue is a product defect, preserve the conversation as product evidence and route it accordingly. The objective is a cleaner system, not simply more content.

    Key takeaways for your next operating review

    • Define self-improvement as a managed loop from conversation evidence to a verified change, not autonomous model learning.
    • Keep resolution rate, resolved volume, coverage, failure reasons, and change throughput visible together.
    • Assign one accountable owner with authority to coordinate support, content, product, and engineering.
    • Classify each failure before fixing it so knowledge, retrieval, behavior, routing, and product problems reach the right layer.
    • Turn repaired failures into regression cases, and apply stronger review as the blast radius increases.
    • Make canonical, AI-ready knowledge a launch requirement instead of a cleanup task for support.

    At your next review, take one recurring unresolved intent and trace it all the way through: evidence, diagnosis, owner, change, evaluation, release, and live result. If any link is missing, that is the first operating gap to repair. Once the path works for one intent, make it the default path for every failure worth learning from.

    References

  • Outcome-Led Product Leadership: A Prioritization System

    Outcome-Led Product Leadership: A Prioritization System

    Your team has more plausible work than capacity. Sales has a customer commitment, support sees recurring friction, engineering sees reliability debt, and executives want a differentiator. Every item can be defended. That is exactly why ranking features is the wrong first move.

    An outcome-led system changes what earns priority. You first decide which customer behavior, product condition, or business result needs to change. Then you compare opportunities and solution bets by how credibly they can cause that change. The roadmap becomes a record of choices, evidence, and trade-offs rather than a queue controlled by the loudest request.

    Prioritize the change before you prioritize the work

    An output is something the team delivers. An outcome is an observable change the team intends to cause. Launching an onboarding flow is an output. Increasing the share of new customers who complete setup successfully is an outcome. The distinction matters because a team can deliver the first without achieving the second.

    A usable outcome needs more than a metric name. It should identify who is affected, what behavior or condition should change, why that change matters, how it will be observed, and which guardrails must remain healthy. If you cannot describe how the world should be different after the work succeeds, the item is not ready to compete for priority.

    Use an outcome card before accepting solution proposals:

    1. Decision context: the strategic problem that makes a choice necessary.
    2. Target population: the customer segment, user role, or workflow affected.
    3. Current state: the observed behavior, baseline signal, or product condition.
    4. Desired movement: the direction of change and, when the evidence supports it, a meaningful target.
    5. Strategic connection: how the change supports growth, retention, trust, efficiency, or another declared priority.
    6. Guardrails: the signals that must not be harmed while the primary outcome improves.
    7. Review trigger: the evidence or constraint change that would cause leadership to reconsider the outcome.

    Do not invent a precise target when no baseline exists. The first commitment may need to be instrumentation, observation, or a small test that establishes the current state. False precision makes an outcome look settled while hiding the most important uncertainty.

    The following layers prevent strategy, outcomes, opportunities, bets, and outputs from collapsing into one roadmap item:

    LayerDecision questionIllustrative setup example
    Strategic intentWhy does this area matter?Make first use dependable for new customers.
    OutcomeWhat observable change should occur?Increase the share of new administrators who finish setup without support.
    OpportunityWhat unmet need or obstacle prevents that change?Administrators cannot tell which permissions are required.
    BetWhat intervention might address the opportunity?Test guided permission configuration.
    OutputWhat would the team actually deliver?Release the validated setup change.

    This separation gives you several places to change course. If the bet fails but the opportunity remains important, try another solution. If evidence shows the opportunity was misdiagnosed, investigate another obstacle. If the outcome no longer supports strategy, stop the entire branch. Without these layers, leaders often preserve a feature commitment long after its original reasoning has failed.

    A company-level result such as revenue can be valid, but it may be too distant for a product team to manage directly. Connect it to customer behavior and product signals the team can influence. Pair each primary signal with a guardrail: setup completion with setup errors, faster resolution with customer-reported quality, or increased usage with reliability. A metric can improve through the wrong mechanism, so success needs a boundary as well as a direction.

    Translate strategy into a decision boundary teams can use

    Outcome-led leadership does not mean selecting a metric and disappearing. Leadership owns the strategic context, the outcome boundary, the investment constraints, and the conflicts that individual teams cannot resolve. The team needs room to investigate opportunities, compare solutions, and stop weak bets without asking permission at every step.

    Training teams in discovery while leaders continue to manage through feature requests, static roadmaps, and approval gates teaches the organization that customer evidence is secondary. Teams may perform interviews and experiments, but they will still optimize for getting a predetermined feature approved and shipped.

    A clear outcome statement can act as a decision boundary:

    For [target segment] in [specific situation], improve [behavior or product condition], observed through [primary signal], because [strategic reason], while protecting [guardrails]. Explore opportunities within [scope and constraints] without assuming [requested solution].

    The last clause is important. A feature hidden inside an outcome statement is still a feature mandate. Improve adoption of the new dashboard assumes the dashboard is the answer. Help account owners notice and act on performance risks leaves room to discover whether a dashboard, alert, workflow change, or no new interface is the better intervention.

    Build a driver tree when the connection between strategy and team behavior is unclear:

    • Place the business result at the top.
    • Identify customer behaviors or product conditions that may contribute to it.
    • Attach observable product signals to those drivers.
    • Map the customer opportunities that could change each driver.
    • Mark every unproven connection as an assumption, not a fact.

    The tree is not proof of causality. It is a visible model of the current reasoning. That visibility helps teams choose what to validate and helps leaders see where a confident roadmap rests on a weak connection.

    Before assigning an outcome, leadership should answer four practical questions:

    • Why does this outcome deserve investment ahead of the alternatives?
    • Which constraints are fixed, and which are merely preferences?
    • Which decisions can the team make without another approval?
    • What evidence would cause leadership to change the outcome or its investment?

    A team cannot genuinely own an outcome when every solution needs executive approval, critical dependencies remain unresolved, or performance is judged only by shipping. That arrangement gives the team accountability without authority. The leadership task is to remove those contradictions before asking the team to move a metric.

    Prioritize opportunities with evidence, then shape the portfolio

    Use an eligibility gate before a ranking formula

    I prefer a gate before a rank. It prevents a polished request with a confident sponsor from competing against a well-understood opportunity merely because both have feature names and effort estimates.

    A candidate should become eligible for prioritization only when its decision brief covers:

    • Outcome relevance: the specific outcome it could affect.
    • Target evidence: the segment, situation, and observed problem behind it.
    • Mechanism: the reason this intervention might change the outcome.
    • Measurement: the primary signal, guardrails, and method of learning.
    • Critical assumption: the belief most likely to invalidate the bet.
    • Constraint fit: the technical, operational, and sequencing limits that matter.
    • Opportunity cost: the work, learning, or outcome investment that would be displaced.
    • Reversibility: the cost of changing course if the assumption proves wrong.

    If a candidate cannot name its outcome or target population, return it to intake. That does not mean it lacks value. It means the organization does not yet have enough information to compare it honestly.

    Scoring models can help expose disagreement, but arithmetic should not make weak evidence look objective. Record the reasoning behind each score. Ask which uncertain input has the greatest effect on the ranking. If a small change to that input reverses the decision, investigate the assumption before committing substantial capacity.

    Compare opportunities before comparing solutions. Several feature requests may be different guesses about the same customer obstacle. Combining them at the opportunity level can reveal a smaller or more effective intervention. Conversely, two similar-looking features may serve different segments and outcomes, which means one score should not flatten them into a false equivalence.

    Use the Kano Model to balance protection, improvement, and exploration

    Outcome relevance tells you why an opportunity matters. The Kano Model adds a customer-expectation lens by separating capabilities into must-haves, satisfiers, and delighters.

    • Must-haves protect the baseline. When they are missing or broken, trust and satisfaction suffer even if the product has innovative features.
    • Satisfiers create more value as their performance improves. Compare the expected incremental outcome movement with the effort and risk required.
    • Delighters create unexpected value and differentiation. Treat them as hypotheses worth testing, not as compensation for a broken baseline.

    Run the classification by segment and context. A capability can be essential for an advanced customer and irrelevant to a new user. Ask how the target customer would feel if the capability existed and how that same customer would feel if it did not. Pairing these functional and dysfunctional questions is more informative than collecting positive reactions to a proposed feature in isolation.

    Do not translate the categories into equal allocations. The right portfolio depends on product maturity, strategic intent, and the condition of the core experience. Make the allocation explicit instead: which investments protect required value, which improve an outcome customers already care about, and which explore future differentiation?

    Revisit the classification after meaningful releases or market changes. A delighter can become an expected baseline, so yesterday’s differentiator may no longer justify the same investment. Usage, experiments, interviews, retention patterns, and support evidence should update the portfolio rather than merely confirm the original roadmap.

    Run leadership reviews that force choices, not status reports

    An outcome-led roadmap can still become output-led in the review meeting. If leaders ask only about delivery dates, scope, and percentage complete, teams will optimize for those signals. Separate the conversations that answer different questions:

    • Outcome review: Is the customer behavior or product condition moving, for which segment, and with what guardrail effects?
    • Discovery review: What changed in the team’s understanding of the opportunity, mechanism, or critical assumption?
    • Commitment review: Which bet should start, continue, change, or stop, and what does that choice displace?

    These conversations can share a meeting, but they should not share one vague status label. On track can mean delivery is proceeding to plan while the underlying evidence is weakening. Healthy delivery and healthy product reasoning are different states.

    Use a compact review board with the outcome and segment, current signal relative to baseline, strongest new evidence, largest unresolved assumption, active bet, decision required, and displaced work. Feature completion belongs in the delivery portion of the review. It should not stand in for evidence that the outcome is becoming more likely.

    Leaders should repeatedly ask:

    • What did the team learn that it did not believe before?
    • Which evidence supports or weakens the proposed mechanism?
    • Is the outcome still right even if the current solution is wrong?
    • What is the smallest next commitment that resolves the most consequential uncertainty?
    • What will stop or move if this work receives priority?
    • Does the team need a decision, a constraint removed, or simply space to continue?

    Set decision conditions before attachment to a solution grows. Continue a bet when the evidence strengthens its mechanism. Change the bet when the outcome and opportunity remain valid but the solution does not. Move to another opportunity when the original problem is weaker than expected. Reconsider the outcome when its strategic premise or target segment changes. Stopping a bet is not abandoning outcome ownership; it is one of the ways outcome ownership becomes real.

    Stakeholder requests need the same discipline. Translate each requested feature into an intake record that identifies the affected customer, the situation, the observed problem, the evidence, the desired behavior change, the timing constraint, and any alternatives already tried. A request earns evaluation, not an automatic roadmap position.

    A useful escalation rule is simple: anyone asking to add committed work must identify what should leave, or explain which outcome or constraint has changed. This turns hidden priority overrides into visible strategy decisions. Seniority may change who has decision rights, but it should not erase opportunity cost.

    Before changing the entire organization, use a pilot team to surface decision bottlenecks, incentive conflicts, stakeholder friction, and policy barriers. Track where the team still needs feature approval, where evidence loses to hierarchy, and where another function is rewarded for behavior that undermines the outcome. Those blockers are leadership work. Scaling the workflow without resolving them only distributes the same conflict more widely.

    Key takeaways for your next prioritization review

    • Prioritize an observable customer, product, or business change before ranking proposed outputs.
    • Give each outcome a target population, baseline signal, strategic connection, guardrails, and review trigger.
    • Separate outcomes, opportunities, solution bets, and outputs so a failed solution does not preserve itself as a permanent commitment.
    • Use an evidence gate before scoring, and expose the assumption that could reverse the ranking.
    • Balance Kano must-haves, satisfiers, and delighters deliberately instead of treating every request as the same kind of value.
    • Make leadership reviews decide what starts, changes, stops, or gets displaced.
    • Convert stakeholder urgency into evidence, constraints, and explicit opportunity cost.

    At your next roadmap review, take the highest-ranked feature and rewrite it as an outcome statement. Require competing bets to name their evidence, critical assumption, guardrails, and displaced work. If the team cannot do that yet, commit to resolving the uncertainty rather than pretending the feature is ready.

    At the following review, ask what changed in the customer signal or the team’s belief before asking what shipped. That question reveals whether your operating system actually rewards outcomes or merely uses outcome language around a feature queue.

    References

  • Product Analytics for Everyone: Master Funnels, Retention, and Conversion to Drive Growth

    Product Analytics for Everyone: Master Funnels, Retention, and Conversion to Drive Growth

    Product analytics isn’t a specialist’s sport—it’s a team capability. In my role leading product teams, I’ve seen designers, engineers, marketers, and customer success partners uncover insights that shape strategy, accelerate product-led growth, and improve outcomes for customers. When we demystify the basics and bring analytics into everyday decisions, we build truly empowered product teams.

    Here’s the core promise of this approach: "Learn the product analytics fundamentals of funnels, retention, and conversion drivers so that anyone can confidently answer key product questions." That line has guided how I teach product managers to think—start with the essentials, tie them to real customer behaviors, and make the work repeatable across the organization.

    I start with funnels because they tell a story—the journey from discovery to value. A simple example: track the path from sign-up to user activation to the first value event. This reveals where onboarding succeeds or stalls, what friction blocks adoption, and which moments are ripe for optimization. With tools like Amplitude analytics or Pendo, we can break down conversions by segment, channel, or feature usage to isolate where improvements matter most.

    Next comes retention analysis, the clearest signal that we’re building something customers choose to return to. Cohort analysis shows who comes back and when; retention curves show where value compels a second, third, and tenth use. Tie retention to activation milestones and the outcomes customers achieve—not just logins—and you’ll quickly spot whether your product discovery assumptions hold up in the wild. A unified analytics platform makes these insights discoverable and repeatable across teams.

    Conversion drivers round out the picture. Once the funnel is clear and retention is stable, I look for the behaviors and experiences that predict success: feature combinations, time-to-value, message timing, or supportive content. Whether in Amplitude analytics or Pendo, correlating these drivers with outcomes lets us prioritize roadmaps with confidence. Pair this with continuous discovery—qualitative interviews, in-product feedback, and rapid experiments—and you’ll move from interesting data to decisive actions.

    This is how we build empowered product teams: by making analytics a daily habit rather than a quarterly report. We bring insights into roadmap reviews, design critiques, and sprint planning; we celebrate learning from experiments as much as shipping features; and we hold ourselves accountable to customer outcomes, not just output. When everyone can interpret funnels, discuss retention, and isolate conversion drivers, we make smarter bets faster.

    If you’re getting started, keep it simple. Define a clear activation metric, instrument the top of your funnel, and track a small number of cohorts. Share a weekly readout with highlights, surprises, and questions to investigate. Over time, stitch insights into narratives that drive product-led growth—and, most importantly, help customers achieve what they came for.

    Product analytics isn’t just for analysts. It’s a shared language for product discovery, onboarding excellence, user activation, and long-term retention. When we practice it together, we build better products and stronger teams.

    Inspired by this post on Amplitude – Best Practices.

    Book a consult png image