Category: Product Management Leadership

  • How to Govern AI Agents With Product Analytics That Drives Action

    How to Govern AI Agents With Product Analytics That Drives Action

    Your dashboard can show growing AI agent usage while the product itself gets worse. Users may invoke the agent, wait for an answer, rewrite it, repeat the task manually, or discover too late that an action needs to be undone. An invocation count records activity. It does not tell you whether the agent was useful, safe, or worthy of more authority.

    If you own an agent roadmap, the practical question is not whether the model can complete an impressive demo. It is whether you can see what the agent did, limit what it was allowed to do, connect its behavior to a user or business outcome, and stop or reverse a bad release. Product analytics should be the control system that helps you answer those questions.

    Key takeaways

    • Define the agent’s job, eligible users, data boundary, action boundary, target outcome, and failure conditions before choosing dashboard metrics.
    • Join product behavior, agent decisions, tool activity, and business outcomes with shared run and workflow identifiers. A model trace or product funnel on its own is incomplete.
    • Treat permissions as product logic. Read access, recommendations, reversible actions, and high-consequence actions need different controls and evidence.
    • Version prompts, retrieval sources, models, tools, policies, and event schemas together so that a change in performance can be traced to a release.
    • Use quality, safety, experience, business, and operational gates to decide whether an agent should expand, remain constrained, be revised, or be retired.

    Define the outcome and authority before the events

    Teams often start by instrumenting what is easiest to count: conversations, messages, tool calls, and thumbs-up feedback. That produces a busy dashboard without a decision model. Start one level earlier. What job is the agent responsible for, and what evidence would justify giving it more reach or authority?

    Write a one-page agent contract

    An agent contract is a product artifact, not a legal document. It creates a stable reference for instrumentation, evaluation, access control, and rollout decisions. Write down:

    • Job: the decision or task the agent helps complete. Avoid broad mandates such as improve support or assist product managers.
    • Eligible workflow: the exact point at which the agent may appear or run. Eligibility must be measurable even when the user never invokes the agent.
    • Eligible users and accounts: the roles, segments, or environments included in the release, plus explicit exclusions.
    • Inputs: the approved resources, fields, retrieval collections, and user-provided context the agent may inspect.
    • Outputs: whether the agent answers, recommends, drafts, updates a system, contacts someone, or triggers another workflow.
    • Human checkpoints: the actions that require review, the person authorized to review them, and what that person must be shown.
    • Target outcome: the user or business result, its denominator, its measurement window, and the system that records it.
    • Known failure states: unsupported answers, irrelevant retrieval, repeated retries, blocked tools, abandoned approvals, incorrect actions, and failed handoffs.
    • Stop condition: the quality, risk, reliability, or outcome signal that pauses the rollout and identifies who owns the decision.

    The eligibility definition matters more than it appears. If you count only people who chose to use the agent, your dashboard excludes people who ignored it, did not notice it, distrusted it, or could not access it. Record the eligible population first. That gives adoption, completion, and outcome metrics a defensible denominator.

    Keep the first contract narrow. A practical starting footprint is one valuable question, a small team, and one assistant. Narrow scope is not merely easier to ship. It makes failures interpretable and limits the consequences of a bad policy, prompt, connector, or event definition.

    Translate authority into enforceable policy

    I use a strict definition of governance: the agent has a bounded objective, a known identity, limited data access, limited tools, recorded policy decisions, an escalation route, and a named owner. A policy page that the runtime cannot enforce is guidance, not governance.

    Authority levelWhat the agent may doEvidence to retainDefault release control
    RetrieveRead approved analytics, records, or knowledge without changing a systemResource identifiers, applied scope, retrieval status, policy version, and references usedPre-approved resources with least-privilege access and data minimization
    RecommendExplain, summarize, rank, draft, or propose an actionAgent version, supporting references, presentation status, and user responseThe user decides whether to accept, edit, reject, or escalate
    Act reversiblyCreate a note or make another bounded change that can be reliably undoneTool, target, before-and-after state, approval, execution result, and reversal pathExplicit approval during the bounded rollout, followed by evidence-based expansion
    Act with high consequenceSend an external communication, alter access or entitlements, disclose sensitive data, or perform a hard-to-reverse operationEverything above, plus approver identity, policy result, purpose, and incident linkageA human makes the consequential decision; eligibility and tool scope remain narrow

    Technical reversibility is not the same as consequence reversibility. A database field may be restored while a customer message, exposed record, or lost trust cannot be recalled. Classify authority by the real-world consequence, not by whether an API offers an undo method.

    Model Context Protocol can make the policy surface clearer because it separates read-only resources from bounded tools and gives agents a standard way to discover them. That interface is useful, but the protocol does not decide who should access a resource, which fields are permitted, or whether an action needs approval. Authentication, authorization, redaction, policy enforcement, retention, and audit logging still belong in your architecture.

    Apply controls before the model call and again before every tool execution. Prompts, retrieved context, logs, and third-party services can all become paths for sensitive-data leakage. Redact data the task does not require, keep secrets outside prompts, use scoped credentials, validate structured tool inputs, and record blocked requests as carefully as successful ones. A denied request is evidence that your policy worked, but repeated denials may also reveal a broken workflow, an overly broad prompt, or an attempted attack.

    Build telemetry that joins agent decisions to user outcomes

    Product analytics and AI observability answer different halves of the same question. A trace can show which context was retrieved, which policy ran, and which tool was called. Product analytics can show what the user did before and after the interaction, which cohort they belonged to, and whether the workflow reached its intended result. Neither view alone proves that the agent created value.

    Join them with two identifiers. An agent run identifier follows one execution from trigger to final status. A workflow identifier connects that execution to the broader task, including manual steps, retries, handoffs, and the eventual business outcome. A user may start several runs inside one workflow, so treating every run as an independent success will inflate apparent demand and hide rework.

    Use a minimum viable event contract

    The following event model is deliberately small. Adapt the names to your analytics conventions, but preserve the states and identifiers.

    Suggested eventRequired propertiesDecision it supports
    agent_eligibleWorkflow identifier, use case, surface, cohort, eligibility reason, and policy versionWho could have used the agent, including people who did not invoke it?
    agent_run_startedRun identifier, workflow identifier, agent version, entry point, and initiating actor typeWhere is the agent being invoked, and how often do workflows require retries?
    agent_answer_presentedRun identifier, answer status, retrieval status, reference status, latency band, and fallback statusDid the user receive a grounded answer, a fallback, or no usable response?
    agent_action_requestedRun identifier, tool, target type, authority level, required scope, approval requirement, and policy resultWhat is the agent attempting, and where are requests blocked or escalated?
    agent_action_finishedRun identifier, tool, execution status, error class, approver state, reversibility state, and duration bandDid an approved action actually complete, fail, time out, or require recovery?
    agent_handoff_startedRun identifier, workflow identifier, handoff reason, destination, context-transfer status, and user choiceWhy did automation stop, and could the receiving person continue without reconstructing the task?
    agent_run_outcomeRun identifier, workflow identifier, completion state, user response, correction state, and failure taxonomyWas the output accepted, edited, rejected, abandoned, retried, or escalated?
    workflow_outcomeWorkflow identifier, outcome name, outcome state, measurement window, and source systemDid the underlying product or business result occur?

    Put the agent, model, prompt, retrieval, tool, policy, and event-schema versions on the relevant records. Without version lineage, a quality shift produces debate instead of diagnosis. You will know that performance changed but not whether the cause was a prompt edit, a new model, a retrieval update, a permission change, a tool release, or broken instrumentation.

    Do not make raw prompts and complete responses the default payload in a general-purpose analytics tool. They can contain personal data, secrets, customer content, or retrieved text that the analytics audience should not see. Send structured classifications and reference identifiers to product analytics. Keep any detailed trace required for investigation in an access-controlled store with explicit retention rules.

    Use enumerated properties for states such as accepted, edited, rejected, blocked, failed, and handed off. Free-text status fields fragment quickly and make reliable cohorts impossible. Preserve a limited diagnostic field only where someone owns its review and classification.

    Measure a stack, not a vanity metric

    A useful scorecard separates five layers. Each layer answers a different management question:

    • Reach and adoption: Of eligible workflows, where was the agent offered and invoked? This shows discoverability and voluntary use, not value.
    • Task experience: Of started workflows, how many completed, retried, fell back, transferred to a person, or were abandoned? Segment edits and overrides instead of treating every acceptance as equally successful.
    • Agent quality: Was the answer supported by approved context, relevant to the request, structurally valid, and consistent with the task-specific evaluation criteria?
    • Governance and safety: Which tool requests were allowed, denied, escalated, or attempted outside the approved scope? Which redaction, moderation, or policy checks failed?
    • Business outcome: Did the downstream result move for the eligible workflow and intended cohort? Examples include completed onboarding, resolved cases, qualified leads, retained users, or a shorter cycle, depending on the contract.

    Always display the numerator and denominator behind a rate. A falling handoff rate may look positive until you discover that completions also fell. A high acceptance rate may hide repeated runs if the dashboard counts only the final answer. A rising task outcome may reflect a changing user mix rather than the agent. Cohort, version, eligibility, and workflow-level views prevent those misreadings.

    Behavioral analytics can establish association and expose where to investigate. It does not automatically establish causality. When the decision requires a causal claim, use a controlled experiment only after both variants meet the same safety and access requirements. Prompts, decision rules, and handoff designs can be tested across appropriate user cohorts; known unsafe behavior, privacy controls, and access boundaries are not experiment variants.

    Turn analytics into release gates, not retrospective reporting

    A governed agent release includes more than a prompt. It includes the model configuration, instructions, retrieval sources, tool definitions, permission scopes, policy rules, user disclosures, approval flow, handoff design, and telemetry. Change any of those and you have changed the product behavior.

    That is why evaluation belongs in delivery, not in a quarterly review. Task-specific test sets, reference answers, error classifications, and pass-or-block thresholds can gate model and prompt changes in CI/CD. Production analytics then checks whether the behavior generalizes to real workflows without weakening the controls established before launch.

    Use a staged promotion path

    1. Validate the interface. Enumerate the resources, tools, schemas, scopes, and denial behavior. Run harmless requests and confirm that unavailable capabilities remain unavailable.
    2. Run task evaluations. Test representative requests, known failure cases, adversarial inputs, missing context, malformed tool arguments, and handoff conditions. Classify failures by consequence rather than relying on one blended quality score.
    3. Exercise the workflow without autonomous consequence. Use dry runs or recommendation-only behavior. Confirm telemetry, references, approvals, fallback, escalation, and rollback before enabling writes.
    4. Release to a bounded eligible cohort. Keep tool scopes narrow and consequential actions under human control. Compare observed behavior with the contract, not with the enthusiasm generated by the demo.
    5. Experiment inside the approved boundary. Test prompt, retrieval, interaction, and handoff variants only after they independently satisfy the safety gate. Analyze results by workflow and version.
    6. Promote or constrain deliberately. Expand access or authority only when the relevant gates pass. A failed safety gate can restrict a release even when adoption or the business metric improves.

    Pre-commit the gates

    Choose thresholds and blocking conditions before reading the launch results. If the team sets them afterward, a promising outcome can quietly lower the quality bar, while a favored feature can turn every failure into an exception.

    GateEvidenceBlocking conditionTypical response
    QualityTask evaluations, grounded-answer checks, correction categories, and unsupported-output reviewsA consequential failure class exceeds the pre-agreed tolerance or lacks a reliable detectorRevise instructions, retrieval, output constraints, or task scope
    Safety and governancePolicy decisions, unauthorized tool attempts, redaction results, approval records, and incidentsAn unresolved high-severity policy or data-control failure remains possibleDisable the affected tool or cohort, rotate credentials where needed, and follow the incident runbook
    User experienceCompletion, edits, rejection, fallback, abandonment, retries, and handoff continuity by cohortThe agent adds work, obscures control, or fails to transfer usable contextSimplify the interaction, improve disclosure, or return the step to a human workflow
    Business outcomeThe contract’s downstream metric for eligible workflows, with an appropriate comparisonUsage grows without a credible improvement in the intended outcomeRevisit the job, target cohort, workflow placement, or value hypothesis
    OperationsTool errors, latency, timeouts, dependency health, fallback success, and rollback readinessThe workflow cannot meet its reliability requirement or cannot fail safelyReduce dependency surface, improve fallback, or pause promotion

    Do not average these gates into a single agent score. A composite score can let strong adoption cancel a serious security failure or let low latency hide poor answer quality. Keep each gate visible, assign its owner, and specify which failures block promotion without negotiation.

    Release decisions should also be reversible. Keep prior prompt, policy, retrieval, and tool configurations identifiable. Define how the runtime disables a tool, narrows a cohort, returns to recommendation-only behavior, or routes directly to a person. A rollback plan that depends on diagnosing the root cause first is too slow for a live incident.

    Make the dashboard an operating system for the product team

    The best agent dashboard does not attempt to show every event. It puts the release decision in view. Organize it in the order the team should reason:

    1. Outcome: eligible workflows, target business result, comparison group where appropriate, and results by cohort and release version.
    2. Journey: eligible, offered, invoked, answer presented, action proposed, approved, executed, handed off, and completed.
    3. Quality and trust: grounded status, acceptance, substantive edits, rejection, retries, corrections, fallback, and qualitative feedback categories.
    4. Governance and operations: allowed and denied tools, approval states, out-of-scope attempts, redaction failures, incidents, errors, latency, and dependency health.

    Every panel should filter by agent version, policy version, tool, entry point, cohort, and workflow outcome. A top-line average is useful for orientation, but releases fail in slices: a user role with missing permissions, a workflow with poor retrieval, a new policy that blocks a required tool, or a handoff destination that cannot use the transferred context.

    Run a decision review, not a dashboard tour

    A regular review with the product trio can use behavioral telemetry, user feedback, and business outcomes to refine prompts, retrieval, and decision logic. Bring security, legal, analytics, operations, or domain owners into decisions that cross their boundaries. The meeting should answer:

    • Which intended outcome moved, for which eligible cohort, and under which release version?
    • Where did users retry, edit, reject, abandon, or request a person, and what does the failure taxonomy show?
    • Which permissions were never needed, and which denied requests reveal either a valid attack defense or a mismatch between the job and the available tools?
    • Did the agent reduce user work, or did it move that work into reviewing, correcting, approving, and recovering?
    • Are outcomes consistent across important roles and workflow entry points, or is the top-line result hiding a weak segment?
    • What changed since the prior release across the model, prompt, retrieval corpus, tools, policies, user experience, and instrumentation?
    • Should the team expand, hold, revise, restrict, roll back, or retire the current behavior?

    Record the decision beside the release lineage: the hypothesis, eligible scope, versions, expected outcome, gates, observed evidence, known risks, owner, and next review condition. This turns governance into an operating history. It also prevents the same debate from restarting when a metric moves or a stakeholder changes.

    Ownership must be explicit. Product owns the job, intended outcome, and promotion decision. Engineering owns runtime reliability, tool boundaries, traceability, and rollback mechanics. Design owns disclosure, user control, approval clarity, correction, and handoff. Data or analytics owns event integrity and metric definitions. Security and legal own the policies and incident requirements within their mandates. Shared input is valuable; shared accountability without a decision owner is not.

    Start with one consequential workflow. Write its contract, add the eligibility event and shared identifiers, classify every available tool by authority, pre-commit the release gates, and review the first bounded cohort against the business outcome. Do not broaden the agent until you can explain why it ran, what it was permitted to see and do, what the user did next, whether the workflow improved, and how you would stop it safely.

    References

  • How to Turn Pendo Adoption Signals Into Revenue Growth

    How to Turn Pendo Adoption Signals Into Revenue Growth

    Your Pendo dashboard can be green while revenue stays flat. Guide clicks, tour completions, and first-time feature use show that something happened inside the product. They do not tell you whether a customer reached value, formed a durable habit, renewed, or became ready to expand.

    A Pendo-led growth motion works only when you connect product behavior to a commercial decision. You need a traceable path from an eligible user, to a valuable behavior, to an account-level change, to an owned go-to-market action, and finally to a revenue outcome. This is how to build that path without mistaking activity for impact.

    Build the revenue path before you build the guide

    Do not begin with a broad goal such as increase adoption. Begin with a decision someone needs to make. Which trial accounts deserve sales attention? Which new customers need onboarding help? Which established accounts show credible retention risk? Which accounts are approaching an expansion conversation?

    For one target segment, write the path in this order:

    1. Commercial outcome: the CRM result you ultimately care about, such as trial conversion, renewal, or expansion.
    2. Eligible cohort: the users or accounts that could reasonably produce that outcome. Exclude employees, test accounts, ineligible plans, and anyone who has already completed the journey.
    3. Value event: the action that represents meaningful progress in the customer’s job, not merely a page view or button click.
    4. Activation milestone: the point at which the user has completed enough of the workflow to experience initial value.
    5. Durable behavior: the repeat usage, adoption depth, collaboration, or seat activity that separates discovery from an established habit.
    6. Commercial trigger: the combination of behaviors that should create a sales, marketing, or customer-success action.
    7. Owner and response: the person responsible, the next action, and the condition that closes or suppresses the signal.

    A generic trial journey might move from connecting data, to completing a core workflow, to returning and repeating it, to inviting colleagues, and then to meeting a defined sales-ready condition. The exact events will differ by product. The discipline is to explain why each event is evidence of customer value and why the final signal should change a commercial decision.

    Time-to-value, feature adoption depth, active usage, and completed trial milestones can help identify purchase readiness. But each metric needs product-specific qualification. Weekly activity is useful only when the workflow naturally recurs weekly. Seat growth is meaningful only when additional users participate in the valuable workflow. A feature click is rarely sufficient evidence on its own.

    Start with one or two high-impact lifecycle plays. Trying to instrument onboarding, conversion, retention, and expansion at once usually leaves every definition open to debate. A narrow pilot forces the team to settle the difficult questions before multiplying them.

    Turn those decisions into a data contract shared by product, growth, RevOps, sales, and customer success. Record the event name, qualifying properties, user and account identifiers, time rule, exclusions, CRM destination, accountable owner, and consent requirements. Define whether an event can occur more than once, how merged identities behave, and what happens when the same person belongs to multiple accounts. Privacy-by-design matters here because behavioral data becomes more sensitive when combined with contact and account context.

    Freeze the definitions for the duration of the pilot. If the activation milestone or eligible population changes after results appear, you no longer have a stable comparison. Log the change as a new version and evaluate it separately.

    Use in-app guidance as a targeted intervention

    Pendo guides are the intervention layer, not the strategy. Their job is to remove a specific obstacle between the eligible user and the next value event. If you cannot name the obstacle and the desired behavior, the guide is likely to become an announcement that generates attention without changing adoption.

    Create a short intervention brief before building anything:

    • Audience: the role, lifecycle stage, account state, and relevant prior behavior.
    • Entry condition: the event or state that makes the message useful now.
    • Friction: the missing knowledge, unclear choice, or incomplete prerequisite preventing progress.
    • Next action: one observable behavior the user can complete.
    • Success event: the downstream product event that counts as progress.
    • Exit condition: the event that permanently stops the guide for that journey.
    • Fallback: help content, support, or human outreach for users who cannot complete the action.

    Match the format to the problem. Use a tooltip when a specific control needs context. Use a short product tour when the user must understand a sequence. Use a banner for broad awareness when an immediate workflow is not required. A modal demands attention, so reserve it for information that justifies interrupting the user.

    Behavioral targeting and progressive disclosure help keep guidance relevant. Show the smallest useful instruction at the decision point, then offer deeper help only when the user requests it or reaches the next step. Suppress the experience as soon as the success event occurs. Repeatedly explaining a completed task trains users to dismiss future messages.

    Test outcome-first copy, placement, calls to action, and guide format, but choose the experiment’s primary outcome outside the guide. A click-through rate can diagnose whether the message earned attention. It cannot establish that the user completed the valuable workflow.

    Define the eligible population before exposure, assign treatment consistently, and select a follow-up window that matches the workflow’s natural cadence. Randomize at the user level when the intervention affects an individual task. Randomize at the account level when colleagues share the experience or one user’s behavior can influence another’s. Otherwise, treatment can leak into the control group.

    Pendo Predict can be used to rank segments by likelihood to convert, expand, or churn. Treat that score as a targeting and prioritization input, not as causal proof. Comparing a high-likelihood group with a low-likelihood group will mostly reveal that the groups were different before the intervention. To learn whether the intervention worked, compare similar eligible users or accounts with and without it.

    Turn product signals into owned revenue actions

    A behavioral signal creates no commercial value while it sits in an analytics dashboard. Connecting Pendo behavior with HubSpot contact and account context makes the signal available inside the workflow where sales, marketing, and customer-success decisions already happen.

    The routing design should answer four questions: What happened? Why does it matter? Who owns the response? When should the signal be ignored or closed?

    Commercial decisionQualifying product evidenceOwned actionSuppression rule
    Trial conversionActivation milestone completed, meaningful feature depth, or a short product-specific time-to-valueRoute the recent behaviors and account context to the sales owner for tailored discoveryExclude internal, test, expired, or already-converted accounts; do not qualify on a guide click alone
    Onboarding recoveryA prerequisite remains incomplete or progress stalls before the value eventCoordinate the next lifecycle message, contextual guide, or customer-success taskStop the journey immediately after milestone completion or confirmed ineligibility
    Retention protectionUse of a core workflow declines relative to the account’s relevant baselineAsk customer success to verify the context before choosing outreach, training, or an in-app interventionDo not label the account as churn risk until role changes, expected inactivity, and other context have been checked
    Expansion qualificationSeat usage grows, more users complete the valuable workflow, or premium capabilities receive meaningful useAsk the account owner to validate the need, entitlement, and buying context before opening an expansion motionSuppress duplicate alerts and activity caused by testing, administration, or temporary access

    Send the evidence behind a signal, not just a label such as hot account or churn risk. The receiving record should include the user and account, triggering behaviors, event timestamps, comparison baseline where relevant, cohort or model version, recommended next action, owner, and current status. If a predictive score is involved, include the behaviors that make the score actionable.

    My rule is simple: if a signal does not change a named person’s next decision, it should not be synchronized yet. Sending every event to the CRM creates noise, duplicate outreach, and mistrust. Send the smallest set of behavioral fields that supports a real decision, then add fields only when an owner can explain how they will use them.

    The same discipline applies to coordinated journeys. An email, chat message, sales task, and in-app guide should not all fire independently from the same behavior. Give the journey one state model so that completing the action in any channel suppresses the remaining prompts. The customer should experience one coherent response, not the internal boundaries between tools.

    Measure incremental lift, not dashboard activity

    Measurement should follow the same chain as the strategy. Keep each stage visible so you can find where performance broke rather than collapsing the journey into a single adoption score.

    • Reach: exposed eligible users divided by all eligible users. This reveals targeting or delivery problems.
    • Guide response: users taking the guide’s intended action divided by exposed users. This evaluates the prompt, not the business result.
    • Activation: eligible users completing the defined milestone divided by the eligible population.
    • Sustained adoption: initial adopters who repeat the valuable workflow during the predeclared follow-up window divided by all initial adopters.
    • Account progression: eligible accounts reaching the defined health, collaboration, usage-depth, or sales-ready condition.
    • GTM response: routed signals that receive the intended owned action, including a documented disposition.
    • Commercial outcome: the relevant CRM result, such as conversion, renewal, or completed expansion, measured at the same entity level as the purchase decision.

    The entity level matters. Guides are often experienced by users, while renewals and expansions happen at the account level. Aggregate user behavior before joining it to an account outcome, and avoid treating multiple exposures inside one account as multiple commercial opportunities.

    Separate influence from incrementality. An influenced account encountered a guide or met a Pendo cohort definition before a commercial outcome. That sequence can support diagnosis and attribution, but it does not establish that the intervention caused the outcome. Incremental impact is the additional result produced compared with what similar eligible accounts would have done without the intervention.

    Use a randomized holdout when the product experience and sample allow it. Declare the primary outcome, minimum effect worth detecting, assignment unit, follow-up window, and stopping rule before launch. Do not stop when an early fluctuation looks favorable. If randomization is impractical, use a staged rollout or a carefully matched comparison cohort, control for concurrent campaigns, and describe the result as directional rather than causal.

    Keep campaign identifiers, guide versions, cohort versions, and event timestamps in the joined dataset. Without them, a launch email, sales outreach, pricing change, and in-app guide can all receive credit for the same outcome. Joining usage cohorts, feedback, lifecycle activity, and pipeline context is useful precisely because it lets you inspect the whole path rather than award credit to the most visible touchpoint.

    At each review, ask where the chain changed. Did the intervention increase activation? Did activation become repeated use? Did account behavior cross the commercial threshold? Did the routed owner respond? Did the CRM outcome move against a credible comparison? Scale only when the evidence survives that sequence. If guide engagement rises but the next product event does not, fix the intervention. If product behavior changes but the commercial result does not, revisit the signal definition or GTM response.

    Key takeaways

    • Choose a revenue decision before choosing a Pendo guide, segment, or dashboard.
    • Define activation as a meaningful value event and distinguish it from discovery, clicks, and first use.
    • Use Predict scores to prioritize attention, then use a valid comparison to measure whether the intervention caused lift.
    • Route only signals that include evidence, an owner, a next action, and a suppression condition.
    • Optimize for sustained behavior and account progression; use guide engagement as a diagnostic metric.
    • Pilot one or two lifecycle plays, stabilize the data contract, and expand only after the full path works.

    For your next rollout, select one commercial question and write its behavioral path before opening the guide builder. Confirm the eligible cohort, success event, control, CRM owner, and exit condition. When every owner can explain the chain in the same terms, Pendo becomes more than an adoption tool: it becomes part of a measurable revenue operating system.

    References

  • A Proven Go-to-Market Playbook: Align ICPs, Positioning, Pricing, Channels, and Launch for Revenue

    A Proven Go-to-Market Playbook: Align ICPs, Positioning, Pricing, Channels, and Launch for Revenue

    I’ve led and learned from dozens of launches, and one truth holds: a sharp go-to-market strategy is the difference between shipping features and creating value. In this piece, I share the playbook I use with my product marketing teams to align product, sales, success, and growth around a single, measurable plan.

    Step-by-step go-to-market strategy for product marketing: Define ICPs, positioning, pricing, channels, launch plan, and metrics to drive adoption and revenue.

    I start by defining our ideal customer profiles (ICPs) with continuous discovery: blending qualitative interviews with quantitative signal from retention analysis and usage. We map jobs-to-be-done, pains, and buying triggers, then size segments and select the entry ICP that maximizes product-market fit odds. From there, we articulate points of parity and competitive differentiation to clarify where we must match the market and where we will win.

    With ICPs locked, I craft positioning and messaging that ladder to a clear value proposition. I test headlines and narratives via A/B testing across ads, email, and in-app guides, and I tighten UX writing inside product tours to reinforce the promise. The goal: consistent, resonant language that sales can champion and self-serve users can understand in seconds.

    Next, I align pricing and packaging to the value metric customers actually care about—keeping SaaS pricing simple to start, with room for advanced consumption SaaS pricing when usage scales. I pair pricing with onboarding that speeds user activation, removes friction with thoughtful tooltip design, and sets customers up for early wins.

    Channel strategy is a focus decision. Depending on motion, I mix product-led growth, targeted outbound, partner co-marketing, and community. I ensure CRM integration and enablement content are ready on day one so marketing, sales, and success can execute in lockstep.

    I translate the strategy into a concrete launch plan tied to product roadmapping and sprint planning: milestones, assets, demos, and a clear owner for every dependency. We rehearse the narrative, pressure-test objections, and equip field teams with competitive battlecards and objection handling.

    From the outset, we define success metrics that ladder to revenue: awareness, activation, conversion, expansion, and retention. Leading indicators beat lagging ones, so I instrument a unified analytics platform to monitor activation rate, time-to-value, and feature adoption in near real time, then feed insights back into the roadmap.

    After launch, we run tight feedback loops—win/loss analysis, in-product surveys, and cohort-based retention analysis—to refine messaging, re-bundle packaging, or adjust channels. The team owns outcomes, not output: we iterate until we see durable signals of product-market fit and efficient growth.

    If you need a simple way to operationalize this, print the one-liner above, share it with your cross-functional partners, and commit to weekly reviews. When everyone can state the ICP, the promise, the price, the channel plan, and the metrics, execution accelerates and the market responds.

    Inspired by this post on Product School.

    Book a consult png image
  • Quantitative Metrics vs. Qualitative Insight: How I Balance Data and Discovery to Grow Products

    Quantitative Metrics vs. Qualitative Insight: How I Balance Data and Discovery to Grow Products

    Quantitative metrics tell the story in numbers; qualitative ones whisper why it matters. Both shape how products grow. Here’s what you need to know.

    In my day-to-day, I rely on quantitative metrics to surface what’s changing in the business and where we need to focus. Activation rate, conversion through the onboarding funnel, feature adoption, retention analysis, and LTV/CAC give me a precise read on performance. I also keep an eye on DORA metrics to understand delivery health and deployment frequency, but I never mistake those for customer outcomes. Numbers spotlight signal—but they rarely explain causality on their own.

    That’s where qualitative analysis earns its keep. Customer interviews, usability studies, win/loss debriefs, support transcripts, and community feedback give me the context behind the charts. Tools like Pendo help me layer in in-app guides and micro-surveys to capture intent and friction in the flow. This combination turns raw data into decisions that actually move the product strategy forward.

    My operating cadence is simple: weekly dashboards to monitor quantitative metrics, ongoing continuous discovery to collect qualitative insight, and a monthly synthesis to reconcile both with our outcomes vs output OKRs. The aim is to move from opinions to evidence, and from anecdotes to patterns. When quant and qual agree, we execute confidently; when they diverge, we design the smallest experiment to learn fast.

    I use a three-question decision tree to choose the method. First, are we exploring or validating? Exploration leans qualitative; validation leans quantitative. Second, do we have enough volume for statistical power? If yes, I’ll run A/B testing with a clear minimum detectable effect (MDE) to avoid false positives. If not, I’ll rely on targeted qualitative discovery until we can instrument a meaningful test. Third, will this decision meaningfully impact our product-led growth or user activation goals? If it will, we invest in both measurement and discovery to reduce decision risk.

    Here’s a concrete example. We once saw a sudden drop in user activation. The quantitative dashboard flagged a step-function change at onboarding step three, but it couldn’t explain why. A quick round of qualitative interviews revealed that our tooltip design buried a critical permission request. We shipped a Pendo-powered in-app guide variant and ran an A/B test to validate the fix. Activation rebounded within a week, and 30-day retention followed suit.

    There are common pitfalls I actively avoid. Chasing vanity metrics that don’t ladder up to outcomes. Conflating shipping speed with customer value by over-indexing on DORA metrics. Overfitting with A/B testing when the MDE is unrealistic for our traffic. And on the qualitative side, mistaking a compelling anecdote for a representative sample without triangulating evidence.

    If you’re looking to tighten your practice, start with a lightweight playbook: instrument core events in Amplitude analytics; define a small set of outcomes vs output OKRs; schedule recurring customer conversations as part of continuous discovery; tag qualitative insights so patterns surface over time; and pair every material UX change with either a well-powered experiment or a clear qualitative learning goal. This creates a unified analytics and discovery loop that compounds.

    Ultimately, quantitative metrics help me prioritize with clarity, while qualitative analysis helps me decide with confidence. When you weave them together, you not only ship faster—you ship the right thing, for the right reason, at the right time.

    Inspired by this post on Product School.

    Book a consult png image
  • Healthcare Product Benchmarks That Matter: Actionable Metrics and Playbooks From Our Report

    Healthcare Product Benchmarks That Matter: Actionable Metrics and Playbooks From Our Report

    I rely on product benchmarks to align teams, sharpen strategy, and accelerate outcomes—especially in healthcare, where stakes are high and complexity is real. Over the years, I’ve learned that the right metrics create clarity across product, engineering, compliance, and go-to-market, enabling faster, safer decisions that translate into measurable impact.

    Discover exclusive data and strategies from our Product Benchmark Report. Compare the healthcare technology industry’s performance across key product metrics.

    When I evaluate a healthcare product’s health, I focus on a few essentials: activation rate and time-to-value for new users, weekly active usage and feature adoption for clinicians and admins, and cohort-based retention analysis to understand whether value compounds over time. I also look at funnel friction (onboarding drop-off, failed setup steps), support load per account, and reliability signals that influence trust—because in healthcare, trust fuels growth.

    Benchmarks turn those metrics into context. They help me answer, “Are we good, or just lucky?” By comparing our numbers to industry peers, I can prioritize the few bets that matter, set outcomes vs output OKRs, and guide empowered product teams to focus on the highest-leverage improvements.

    Operationally, I instrument products with a unified analytics platform and tools like Amplitude analytics and Pendo to track user activation, feature adoption, and in-product journeys. Pairing that with continuous discovery keeps insights fresh, while A/B testing and clear minimum detectable effect (MDE) thresholds ensure we ship with statistical confidence.

    In practice, my playbook for healthcare product-led growth is straightforward: simplify onboarding with targeted product tours and in-app guides, tighten the first-win loop to reduce time-to-value, and eliminate blockers surfaced by behavioral analytics. Then, reinforce the loop with lifecycle messaging, role-specific education, and clear value propositions for clinicians, operations teams, and executives.

    Of course, none of this works without strong governance. Data governance and regulatory compliance aren’t just guardrails; they’re growth enablers. Clear audit trails, privacy-by-design, and reliable incident management build the trust that keeps adoption high and churn low.

    If you’re ready to benchmark your roadmap against the market, this report gives you the clarity to spot gaps, the language to align stakeholders, and the metrics to execute with precision. Use it to calibrate your product strategy, guide your next set of experiments, and confidently scale what works across the healthcare technology ecosystem.

    Inspired by this post on Amplitude – Perspectives.

    Book a consult png image
  • The New AI Playbook for Product Portfolio Optimization: Slash Complexity, Boost ROI

    The New AI Playbook for Product Portfolio Optimization: Slash Complexity, Boost ROI

    The most valuable lesson I’ve learned leading product organizations is that portfolio choices make or break outcomes. In an era of infinite requests and finite teams, the question isn’t what we could build—it’s what we must build next. That’s why I’m codifying a pragmatic, AI-driven playbook to optimize the product portfolio while staying true to outcomes, not output.

    AI-powered product portfolio optimization is here. Explore strategies and tools helping product leaders manage complexity and boost ROI.

    My starting point is a data backbone that connects strategy to reality. I aggregate product usage, revenue by segment, cost-to-serve, retention cohorts, and support signals into a unified analytics platform, then layer a retrieval-first pipeline so LLMs can reason over clean context. Instrumentation matters: Amplitude analytics, Pendo, and in-app guides provide the behavioral and activation signals that make prioritization measurable.

    From there, I translate strategy into an objective decision system. I express outcomes vs output OKRs, align initiatives to value proposition and competitive differentiation, and classify opportunities with the Kano Model. LLMs for product managers help cluster voice-of-customer at scale; with thoughtful prompt engineering and AI workflows, I can map themes to jobs-to-be-done, quantify demand, and de-duplicate asks across stakeholders.

    Execution hinges on evidence. I run A/B testing with a clear minimum detectable effect (MDE), pair it with eval-driven development for AI features, and ship through CI/CD while tracking DORA metrics. This closes the loop between product roadmapping and sprint planning and real-world performance—activation, retention analysis, and Web Vitals inform the next set of portfolio bets.

    Trust is a feature, so governance is built-in. Privacy-by-design, data governance, and AI risk management guide how we store, prompt, and evaluate models. I apply guardrails to sensitive workflows and define success metrics that balance short-term ROI with long-term resilience and regulatory compliance.

    The operating model matters as much as the models themselves. Product trios and empowered product teams run continuous discovery, pressure-test assumptions in QBRs vs OKRs, and make trade-offs visible. Stakeholder management becomes easier when the portfolio narrative is anchored in transparent scenarios and shared metrics.

    If you’re getting started, here’s my flow: unify data, define outcomes, segment opportunities, simulate scenarios, and test fast. Use LLMs to synthesize signals you’d never humanly read, then make one focused bet per team that moves a measurable KPI. Rinse, learn, and reallocate—portfolio optimization is a living system, not an annual meeting.

    Ultimately, the promise of this new playbook is simple: less noise, sharper focus, and compounding ROI. By pairing AI Strategy with disciplined product management leadership, we can manage complexity with clarity—and consistently build what matters most.

    Inspired by this post on Product School.

    Book a consult png image
  • Game-Changing Product Benchmarks Every Media & Entertainment Leader Must Know

    Game-Changing Product Benchmarks Every Media & Entertainment Leader Must Know

    Benchmarks are my reality check. In the fast-moving media and entertainment space, I rely on concrete product metrics to align strategy, prioritize roadmaps, and drive product-led growth with confidence. When my team and I calibrate against industry benchmarks, we turn opinions into outcomes and ensure our bets are tied to measurable impact.

    Discover exclusive data and strategies from our Product Benchmark Report. Compare the media and entertainment industry’s performance across key product metrics.

    Here’s how I think about what matters most in this report: user activation and time-to-value to understand onboarding effectiveness, retention analysis to quantify staying power, feature adoption to validate value delivery, and engagement depth to see whether we’re building habit loops—not just generating clicks. I also look at experimentation maturity (A/B testing volume and velocity), release cadence, and how we structure outcomes vs output OKRs to keep teams accountable to real customer impact.

    Benchmarks aren’t scorecards—they’re decision accelerators. I use them to run a gap analysis, set clear targets, and focus the roadmap on the few bets most likely to move our leading indicators. For example, if activation lags, we invest in clearer in-app guides, product tours, and progressive onboarding; if retention stalls, we refine the value proposition and instrument cohorts to isolate which segments respond best.

    Operationally, I instrument a unified analytics platform with Amplitude analytics for cohorting and funnel analysis, and Pendo for in-app guidance and feature adoption insight. Weekly product health reviews keep the team oriented around activation, retention, and engagement. When we A/B test, we set a minimum detectable effect (MDE) up front and tie experiments to specific OKRs, so decisions aren’t swayed by noise. This discipline helps empowered product teams ship faster without sacrificing rigor.

    If you’re building in media and entertainment, use these benchmarks to define what “good” looks like for your model, then localize targets to your audience and content format. Start by instrumenting the essentials, align leaders on the few metrics that matter, and iterate with high-velocity experiments. The right benchmarks will sharpen your product strategy, improve stakeholder confidence, and turn your roadmap into a reliable engine for growth.

    Inspired by this post on Amplitude – Perspectives.

    Book a consult png image
  • How to Build an Amplitude-Led Product and Content Loop

    How to Build an Amplitude-Led Product and Content Loop

    If your Amplitude workspace contains more dashboards than decisions, you do not have an analytics problem. You have an operating-model problem. Marketing improves clicks, product optimizes activation, and lifecycle content ships on a calendar, but nobody can show which message changed a valuable user behavior.

    An Amplitude-led growth loop connects observed behavior to a content decision, a measurable intervention, and a later product outcome. The goal is not more reporting. It is a repeatable way to decide what to say, where to say it, who should see it, and whether it created durable value.

    Key takeaways

    • Start with a user journey and a pending decision, not a request for another dashboard.
    • Treat landing-page copy, onboarding instructions, product tours, in-app guides, and lifecycle messages as product interventions with intended behavioral outcomes.
    • Use funnels to locate friction, behavioral cohorts to compare paths, and retention analysis to test whether an activation gain lasts.
    • Instrument eligibility, assignment, exposure, and outcome separately so you know who could have seen the content and who actually did.
    • Set the primary metric, guardrails, minimum detectable effect, and decision rule before reviewing experiment results.

    Start with the growth decision, then design the measurement

    A unified analytics platform is only useful when it shortens the distance between a question and a decision. Before opening Amplitude, write the decision your team expects to make. A useful decision is concrete: change an onboarding step, reposition a capability, trigger an in-app guide later, stop a lifecycle message, or invest in a product-tour pattern.

    Create a one-page measurement contract for the journey:

    1. User outcome: State what the person is trying to accomplish in their language, not the name of your feature.
    2. Eligible population: Define the lifecycle stage, role, account condition, prior behavior, and acquisition context that make someone part of the decision.
    3. Activation behavior: Name the observable action that indicates the user reached initial value. Do not automatically substitute registration, a page view, or a content click for value.
    4. Content intervention: Identify the message or guidance you are prepared to change and the moment when it can affect the next decision.
    5. Primary outcome: Choose the downstream behavior that will determine whether the intervention worked.
    6. Decision rule: Write what you will ship, revise, or stop for each credible result, including an inconclusive result.

    Keep four metric types separate. A North Star metric aligns the organization around delivered customer value. An activation metric identifies an early value moment. A diagnostic metric, such as guide completion or a call-to-action click, helps explain the path. A guardrail catches an unwanted tradeoff, such as more setup completion followed by weaker retained usage. A content click can be useful without deserving promotion to the North Star.

    Your event specification should define the behavior, actor, account, surface, content version, relevant context, and trigger condition. Use stable user and account identities across the website, CRM, and product wherever your governance model permits it. If an anonymous visitor becomes an authenticated user but the identities are not reconciled, the funnel can manufacture a drop-off that did not occur. In a multi-user product, decide whether value belongs to a person, an account, or both before building cohorts.

    Validate the instrumentation by performing the real journey and inspecting the resulting sequence. Check that events fire once, required properties arrive, content versions are distinguishable, and excluded users remain excluded. If a metric cannot change a product or content decision, remove it from the working view. Dashboard completeness is not the goal; decision readiness is.

    Read behavior as a content problem you can test

    Funnels, cohorts, and retention views answer different questions. A funnel tells you where progression breaks. A behavioral cohort lets you contrast users who reached value with those who did not. A retention view shows whether the behavior associated with activation continues. The useful insight usually appears when you combine them rather than treating any one chart as the verdict.

    Do not jump from a drop-off to a copy rewrite. Analytics shows what people did; it does not, by itself, prove why they did it. Convert the signal into a falsifiable content hypothesis, then choose the intervention closest to the decision that appears to be failing.

    Behavioral signalWorking hypothesisContent action to testOutcome to inspect
    Users begin setup but leave before completing the first meaningful configurationThe step asks for information before explaining its purpose or expected resultClarify the outcome, required inputs, and next step at the point of setupConfiguration completion followed by the activation behavior
    Users reopen the same guide but do not perform its next actionThe guidance explains a concept without resolving the immediate taskReplace general explanation with the exact next action and contextual helpProgression to the intended product event, not guide opens
    A lifecycle message earns clicks but recipients do not reach value in the productThe promise, audience, or destination does not match the recipient’s readinessAlign the message with the prerequisite behavior and the correct in-product destinationPost-click activation among eligible recipients
    Retained users adopt a capability after a recognizable prerequisite sequence, while new users rarely find itThe capability is useful but introduced before the user has enough contextTrigger an in-app guide after the prerequisite sequence rather than during initial onboardingQualified adoption and later retained usage

    The location of the intervention matters. Use website content to set an accurate value proposition. Use onboarding copy and empty states to help a new user make the next necessary decision. Use a product tour when the sequence itself needs orientation. Use a contextual guide when prior behavior indicates readiness. Use CRM content to bring the person back to a specific unfinished or newly relevant task. Behavioral cohorts can connect these surfaces to the same product lifecycle instead of leaving each channel with its own definition of success.

    Give every content asset a measurable job. Record its audience, lifecycle stage, trigger, intended next behavior, primary outcome, owner, and retirement condition. Content without a distinct job accumulates because nobody can prove that it is redundant. Content with a defined job can be improved, reused, or removed.

    Targeting also needs restraint. Collect only the identity and behavioral properties required for the decision, govern access to them, and avoid sensitive segmentation that the use case does not require. Privacy-by-design and consistent information architecture are part of a trustworthy content system, not cleanup tasks for after growth work succeeds.

    Run content experiments with product-level discipline

    Once content is tied to an observable behavior, test it with the same discipline you would apply to a product change. The experiment brief should fit on one screen, but it needs enough precision that another person could reproduce the analysis.

    • Hypothesis: For a defined eligible group, changing a specific surface from the current experience to a proposed experience should affect a named behavior because of a stated mechanism.
    • Eligibility: Define who can enter the experiment and what prior behavior qualifies them.
    • Control and treatment: State exactly what differs. If audience, timing, placement, and copy all change together, you will not know which mechanism mattered.
    • Assignment and exposure: Record assignment independently from actual exposure. A person assigned to a guide but never shown it should not be mistaken for someone who saw and ignored it.
    • Primary metric: Use the closest meaningful product outcome that the content is intended to affect.
    • Diagnostics and guardrails: Track intermediate behavior for explanation and downstream behavior for unintended effects.
    • Decision parameters: Set the minimum detectable effect, analysis population, reading window, and stopping condition before looking at the result.

    The minimum detectable effect is the smallest change that would be worth detecting and acting on. It belongs in planning because it shapes the sample requirement and determines whether the experiment can answer the business question. Sizing the MDE and aligning on success metrics before launch prevents a weak test from becoming a confident story after the fact.

    Watch for five common analytical traps:

    1. Optimizing the content interaction: A higher click-through or tour-completion rate is not a win if activation does not move.
    2. Logging assignment as exposure: This dilutes the measured effect when eligible users never encounter the intervention.
    3. Reading every segment after the result: Unplanned slicing can produce an attractive pattern that does not hold up. Treat it as a new hypothesis.
    4. Stopping when the chart looks favorable: Repeatedly checking and ending a conventional fixed-horizon test early weakens the reliability of the conclusion.
    5. Forcing a winner: A result can support the treatment, support the control, or remain inconclusive. The third outcome is a valid decision state.

    Low traffic does not justify lowering the evidentiary standard while keeping the same confident language. You can test a clearer contrast, wait for a suitable observation window, narrow the decision, or combine genuinely equivalent surfaces when they represent the same hypothesis. If you proceed without a powered experiment, label the result as directional and keep causal claims modest.

    Make each result change the product-content system

    An experiment creates value only when its result changes what happens next. End every readout with a decision record containing the original signal, eligible cohort, hypothesis, intervention, metric definitions, result, limitations, owner, and next action. Link that record to the dashboard, event specification, content version, and release. This prevents a later team from repeating the test under a different name.

    Keep product, design, engineering, content, and lifecycle owners on one instrumentation plan. A shared plan across the people designing the product and its guidance keeps the website promise, in-product experience, and follow-up message tied to the same user outcome. It also makes ownership explicit when the problem is not copy: content cannot repair a broken workflow, missing capability, or inaccessible destination.

    Use a recurring decision cadence built around one journey at a time:

    1. Select a valuable journey with visible friction and an owner prepared to change it.
    2. Verify the event sequence and identity model before interpreting the funnel.
    3. Compare the stalled cohort with a cohort that reached value, then inspect differences in sequence, context, and prior behavior.
    4. Write the content hypothesis and choose the surface nearest the failed decision.
    5. Confirm experiment readiness, including exposure tracking, MDE, guardrails, and the later retention window.
    6. Ship the intervention, read the result against the original decision rule, and record the decision.
    7. Scale the pattern only where audience, trigger, mechanism, and intended outcome still match.

    Do not stop at immediate activation. Revisit the eligible control and treatment cohorts over a retention window appropriate to your product’s natural usage cycle. If the treatment increases an early action but retained usage stays flat or weakens, the content may be accelerating shallow completion rather than helping users reach durable value. Investigate that mechanism before rolling the pattern across onboarding or lifecycle campaigns.

    Your next move is deliberately small: choose one stalled journey, write the decision you need to make, and validate the event sequence before opening another dashboard. Then ship one content intervention whose exposure and downstream outcome you can measure. That is enough to start turning Amplitude from a reporting destination into a product and content growth loop.

    References

  • Enterprise Go-To-Market That Wins: How Product Marketing Supercharges Analytics Adoption

    Enterprise Go-To-Market That Wins: How Product Marketing Supercharges Analytics Adoption

    In my role leading product management at HighLevel, I’ve learned that enterprise go-to-market lives or dies by the strength of the partnership between product and product marketing. When we operate as one team, we turn complex capabilities into clear outcomes that resonate with buyers and drive adoption at scale.

    I’m especially energized by the archetype of a product marketing manager at a leading analytics platform—someone “focusing on go-to-market solutions for enterprise customers.” That mandate requires rigor across product positioning, value proposition design, competitive differentiation, and sales enablement, all while aligning deeply with engineering and customer success. In practice, it means translating signal from a unified analytics platform into narratives and plays that close deals and expand accounts.

    Day-to-day, I partner with product marketing to validate messaging through continuous discovery and data. We use Amplitude analytics to instrument activation, engagement, and retention analysis—then feed those insights into product-led growth motions like in-app guides and product tours. A/B testing grounded in a clear minimum detectable effect (MDE) helps us separate noise from impact, while points of parity and true differentiation shape the story sellers can confidently carry into enterprise conversations.

    This is also where outcomes vs output OKRs keep us honest. Rather than celebrating launches, we anchor on measurable behavior change: faster time-to-value, higher user activation, deeper feature adoption, and multi-threaded stakeholder engagement. Product trios provide the operating rhythm, and stakeholder management ensures sales, marketing, and success move in lockstep with the roadmap and GTM calendar.

    If you’re building an enterprise GTM motion, start by tightening your value proposition to the top three pains your best-fit accounts actually feel, validate with real usage data, and then enable your field teams with crisp, data-backed talk tracks. With the right PM–PMM alignment and analytics foundation, your go-to-market strategy becomes a compounding advantage—not just a launch plan.

    Inspired by this post on Amplitude – Perspectives.

    Book a consult png image
  • How to Connect Voice of Customer to Behavioral Analytics

    How to Connect Voice of Customer to Behavioral Analytics

    You have interview notes, support tickets, sales objections, app reviews, and in-product feedback. Yet the roadmap discussion still comes down to which customer complained most recently or which stakeholder tells the most persuasive story.

    The way out is not another survey. Connect each voice-of-customer theme to the behavior of the people who expressed it. You can then see whether the problem changes activation, task completion, adoption, retention, or conversion; identify where the friction occurs; and decide whether the opportunity deserves roadmap space.

    Start with the decision, not the feedback backlog

    VOC becomes useful when it can change a decision. Before analyzing a theme, ask what you would do differently if the concern proved material. Would you redesign an onboarding step, improve reporting performance, simplify permissions, clarify pricing, or leave the current experience alone?

    If the answer is unclear, the theme is not ready for prioritization. It may still be worth tracking, but it should not become a roadmap item merely because it appears frequently.

    Write the theme as a behavioral hypothesis:

    Customers who encounter or mention [theme] while attempting [job] are more or less likely to [observable behavior] within [relevant window] than comparable customers who do not.

    VOC-to-behavior hypothesis template

    A useful hypothesis contains six parts:

    • Population: The users or accounts eligible to encounter the problem.
    • Job: What they were trying to accomplish, not merely the page they visited.
    • VOC theme: The friction expressed in neutral language, such as onboarding confusion or performance slowness.
    • Behavioral signal: The action or pattern you expect to observe, such as abandonment, backtracking, repeat clicks, or slow task completion.
    • Outcome: The activation, adoption, conversion, or retention metric that could move.
    • Window: The period in which that behavior and outcome are meaningful for your product.

    For example, a complaint that a flow is too complex can become a testable expectation: affected users will take longer on a step, move backward more often, depend more heavily on tooltips, or abandon the funnel at a particular screen. Those observations will not explain the customer’s motivation on their own, but they will reveal whether the stated friction has a visible behavioral footprint.

    This distinction matters. Feedback explains how customers interpret an experience. Analytics records what happened. Neither is sufficient alone. Treat the comment as a hypothesis and observable product behavior as the evidence that tests it.

    Build a shared spine between what customers say and do

    You cannot reliably connect VOC to behavior when the two systems describe customers, product areas, and outcomes differently. The work begins with a shared measurement spine: consistent identities, timestamps, product concepts, and definitions.

    Instrument the moments that represent value

    Do not begin by tracking every click. Begin with the moments that determine whether a customer reaches value:

    • The start and end points used to calculate time-to-first-value.
    • The steps and completion event in the onboarding funnel.
    • The first meaningful use of a core feature.
    • The repeated behaviors that indicate adoption rather than experimentation.
    • The conversion event that represents a real commitment.
    • The activity and return criteria used in retention analysis.

    Each event needs an explicit trigger, a user or account identity, a timestamp, and the contextual properties required for segmentation. In a business product, retain both user-level and account-level identity where your data rules permit it. A frustrated user may submit the ticket, while account retention and revenue are measured elsewhere.

    Definitions deserve the same discipline as instrumentation. If onboarding completion means reaching one screen to Product and completing a different workflow to Customer Success, the resulting cohort comparison will settle nothing. Record the definition, owner, applicable population, and known exclusions for every decision metric.

    Amplitude analytics, Pendo, or another unified analytics platform can support funnels, cohorts, and retention curves. The platform does not remove the need for a clean event taxonomy. Better charts built on inconsistent events only make the wrong conclusion look more convincing.

    Normalize VOC without stripping away its meaning

    Customer feedback arrives in incompatible forms: a support ticket describes a blocked task, a sales note records an objection, an app review compresses several problems into one comment, and an in-product response refers to the screen the customer is currently viewing. A shared theme taxonomy makes those inputs comparable.

    For each feedback record, capture the minimum fields needed to analyze it:

    • The original wording or a reference to it, so the nuance remains recoverable.
    • A neutral theme and, where necessary, a more specific subtheme.
    • The product area and job the customer was attempting.
    • The date, touchpoint, and customer or account identifier available under your privacy and data-governance rules.
    • The customer’s lifecycle stage, plan, role, or other context needed to define an eligible comparison group.
    • Whether the customer described a symptom, proposed a solution, or did both.

    That final distinction prevents a common roadmap error. A request for another button is a proposed solution. The underlying problem may be that the current action is hard to discover, too slow, or unavailable to the customer’s role. Preserve the request, but tag the friction separately. Otherwise, you will count preferred implementations rather than customer problems.

    Keep the taxonomy small enough that different people apply it consistently. Split a theme only when the distinction would produce a different cohort, root-cause investigation, or product decision. A label that never changes analysis is administrative detail, not useful structure.

    Turn each VOC theme into a fair cohort comparison

    Once the datasets share identities and definitions, build a cohort containing the users or accounts associated with a theme. Then compare that group with customers who were genuinely capable of encountering the same experience.

    Use this sequence:

    1. Define the expressed cohort. Include customers associated with the theme during a stated period. Preserve the feedback date so you can distinguish behavior before and after the comment.
    2. Define eligibility. Exclude customers who could not access the feature, workflow, plan, permission level, or product version involved.
    3. Create the comparison cohort. Use customers with a similar lifecycle stage and opportunity to perform the job, but without the same recorded theme.
    4. Align the observation window. Give both cohorts the same opportunity to complete the funnel, activate, adopt the feature, or return.
    5. Locate the behavioral difference. Compare funnel steps, task time, navigation patterns, feature adoption, conversion, and retention where each is relevant.
    6. Segment the result. Check whether the effect is concentrated by role, plan, account type, entry path, or another product-relevant dimension.
    7. Return to the qualitative evidence. Review the wording and relevant sessions around the point where behavior diverges. This is where the probable cause becomes specific enough to design against.

    The comparison group matters as much as the expressed cohort. Users who contact support are not a random sample. They may be more engaged, more experienced, more valuable, or simply more willing to report problems. A behavioral difference therefore shows an association worth investigating; it does not prove that the theme caused the outcome.

    Timing creates another trap. A customer may open a ticket because a task already failed. If you combine activity from before and after the ticket, the analysis can confuse the cause, the failure, and the attempt to recover. Anchor the timeline to the relevant exposure or task attempt, and use the feedback timestamp as context rather than automatically treating it as the beginning of the problem.

    Interpret repeated actions carefully as well. Repeat clicks can indicate an unresponsive control, uncertainty about whether a request registered, or deliberate power use. Backtracking may reflect confusion or a legitimate comparison workflow. Pair the pattern with funnel position, timing, interface state, and customer language before naming the root cause.

    Your output should be an evidence statement, not a dashboard tour. A strong statement identifies the eligible segment, the observed difference, where it appears, the outcome associated with it, and the remaining uncertainty. That is enough for a product trio to decide whether to investigate, intervene, or stop.

    Prioritize the behavioral gap and validate the fix

    Raw feedback volume is a weak prioritization rule because it has no denominator. A theme can generate many tickets because the workflow is widely used, because the problem is severe, or because the affected customers are unusually vocal. Reach, behavioral impact, and proximity to a meaningful outcome separate those possibilities.

    Build a compact opportunity case for each material theme:

    • The eligible population and the portion associated with the theme.
    • The behavior gap between the expressed and comparison cohorts.
    • The funnel, activation, adoption, conversion, or retention outcome connected to that gap.
    • The segment in which the effect is concentrated.
    • The probable root cause and the evidence supporting it.
    • The smallest intervention capable of testing that cause.
    • The primary metric, guardrails, and uncertainty that remain.

    A practical sizing model is: eligible population multiplied by the observed behavior gap multiplied by the value of recovering the affected outcome. Use a range when the inputs are uncertain. The purpose is not to manufacture a precise forecast. It is to expose whether your business case depends on broad reach, a large outcome gap, a valuable segment, or an assumption that still needs evidence.

    Do not rank opportunities by the size of the gap alone. A large drop in a low-value side path may matter less than a smaller gap immediately before activation. Conversely, a retention difference may be associated with the theme without being caused by it. Confidence intervals and explicit assumptions help keep opportunity sizing proportional to the evidence.

    When you ship, test the causal claim you actually care about. State the eligible population, intervention, primary metric, guardrails, and minimum detectable effect before looking at results. Use an A/B test when random assignment is practical. If you must rely on a staged rollout or observational comparison, label the result accordingly and keep plausible alternative explanations visible.

    Success is not a warmer survey response by itself. The behavior implicated by the original theme should move: fewer relevant drop-offs, less unnecessary backtracking, faster task completion, stronger activation, or better retention. Sentiment can confirm that the experience feels better, but the original behavioral hypothesis should still be tested.

    What a complete feedback-to-outcome loop looks like

    One reporting experience illustrates the sequence. Customers described reporting as slow. The behavioral trail contained long load times and repeated clicks on filters, which narrowed the problem beyond the broad complaint. The response combined simpler defaults, prefetching important queries, and clearer loading states. In that case, the changes reduced perceived wait time by 42% and improved day-7 retention for the affected cohorts.

    That result is a case-specific outcome, not a benchmark to paste into another business case. The transferable lesson is the chain of evidence: customer language identified the experience, behavioral data located the friction, the intervention addressed the probable mechanism, and the affected cohort supplied the right place to measure retention.

    Make this chain part of the operating cadence. Use a weekly listening review with the product trio to classify emerging themes and flag missing instrumentation. Use a monthly synthesis to join mature themes with usage data, refresh opportunity cases, and retire claims that behavior does not support. When a change ships, return to the original expressed cohort and the relevant outcome window rather than declaring success from aggregate usage.

    Key takeaways

    • Start with the roadmap decision a VOC theme could change, then express the theme as a behavioral hypothesis.
    • Give feedback and product events a shared spine: consistent identities, timestamps, product areas, jobs, and outcome definitions.
    • Compare customers who expressed a theme with customers who had the same opportunity to encounter the experience.
    • Align observation windows and lifecycle stages before interpreting funnel, activation, adoption, or retention differences.
    • Treat cohort differences as evidence of association, not automatic proof of causation.
    • Prioritize the affected population, behavior gap, outcome value, and strength of evidence rather than ticket volume alone.
    • Validate the proposed mechanism with an experiment and a predetermined minimum detectable effect whenever random assignment is practical.

    At your next listening review, choose the VOC theme consuming the most roadmap attention. Write one behavioral hypothesis, identify the eligible cohort, and compare one outcome that would make the problem worth solving. If you cannot complete that chain, the next priority is not another feature request. It is the missing identity, event, definition, or feedback tag preventing you from making the decision responsibly.

    References

  • Inside Google’s Product Model: Hard-Won Lessons to Build Empowered, Outcome-Driven Teams

    Inside Google’s Product Model: Hard-Won Lessons to Build Empowered, Outcome-Driven Teams

    I’ve been systematically exploring how the product model shows up inside iconic companies. After studying “The Product Model at Spotify” and “The Product Model at Amazon,” I’m turning my lens to Google—specifically, how the product operating model, product culture, and product strategy manifest in practice and what we can pragmatically take back to our own organizations.

    When I talk about the product model, I’m looking at the machinery that connects strategy to outcomes: empowered product teams, clear decision rights, tight product trios, continuous discovery, data-informed bets, and an operating cadence that enables learning at speed. My goal here is to unpack how those elements come together at Google and translate them into repeatable patterns you can adopt.

    At a high level, I focus on how teams are empowered to solve problems rather than ship outputs, how outcomes vs output OKRs clarify what matters, and how experimentation (from rapid prototyping to A/B testing) de-risks decisions before they scale. I also examine how engineering and product partner to balance platform scalability with customer value, and how stakeholder management reinforces alignment without slowing teams down.

    Why does this matter? Because the product model is a lever for resilience and speed. When product strategy is explicit and the operating model is built for learning, organizations multiply the impact of talented people. That’s how small, focused teams repeatedly deliver outsized results—even in complex, regulated, or high-scale environments like Google.

    In the sections that follow, I’ll synthesize what I see as the core patterns behind Google’s approach and distill them into actionable guidance: how to structure product trios, how to run continuous discovery alongside delivery, how to set and calibrate OKRs for outcomes, and how to evolve your product culture so empowered product teams can do their best work. My aim is not to idolize a model, but to extract what’s portable and help you adapt it to your context.

    Inspired by this post on SVPG.

    Book a consult png image
  • Retail and Ecommerce Product Benchmarks That Drive Growth

    Retail and Ecommerce Product Benchmarks That Drive Growth

    You probably don’t need another ecommerce dashboard. You need to know whether a weak number represents a real customer problem, a measurement defect, or a change in the mix of people visiting your store.

    That distinction matters because each diagnosis leads to a different roadmap. A benchmark can help you find the gap, but it cannot explain the gap or choose the response. This framework shows you how to move from an external comparison to a defensible product decision, a clean experiment, and a measurable business outcome.

    Use the benchmark to frame one decision

    A benchmark is context, not a target. Used well, benchmarks connect acquisition, activation, conversion, retention, and unit economics so you can see where the customer journey is underperforming. Used poorly, they turn into arbitrary goals that ignore your customer mix, business model, and measurement definitions.

    Start by writing a benchmark brief in one sentence:

    For this customer segment, compare this precisely defined metric over this observation window so I can make this product decision.

    That sentence forces four questions into the open:

    • Who is included? New or returning customers, mobile or desktop users, and subscription or one-time buyers can behave differently.
    • What exactly is counted? A visit, person, cart, order, and subscription are different units. Pick the unit before you calculate the rate.
    • When does the observation end? Conversion can be measured immediately, while repeat purchases, returns, refunds, and subscription retention need time to mature.
    • What decision will change? If a better or worse result would not alter the roadmap, experiment, or allocation of attention, the comparison is decorative.

    Build the scorecard around the customer’s journey rather than the structure of your organization. This prevents marketing, product, commerce, and customer experience teams from presenting separate versions of performance.

    Journey stagePrimary metricUsable definitionDecision it should inform
    AcquisitionVisit-to-signupCompleted signups divided by eligible visits, when account creation is a meaningful part of the journeyWhether the arrival experience and value proposition earn the next commitment
    ActivationTime-to-first-valueElapsed time from a defined starting event to a customer action that represents real valueWhether onboarding helps a new customer reach a useful outcome without avoidable delay
    ConsiderationProduct-to-checkout conversionCheckout starts divided by qualified product viewersWhether customers move from evaluating a product to expressing purchase intent
    CheckoutOrder completion rateCompleted orders divided by checkout startsWhether the transactional flow converts existing intent into an order
    RetentionRepeat purchase or subscription retentionEligible customers who purchase again, or subscriptions that remain active, within a defined observation periodWhether value continues after the first transaction
    EconomicsAverage order value and LTV/CACRevenue per order, and customer lifetime value relative to customer acquisition cost, using documented revenue and cost definitionsWhether growth creates sufficient customer value and business value
    FrictionCart abandonment, return rate, and refund rateClearly scoped failure or reversal events tied to the relevant cart, order, or customer cohortWhether an apparent conversion gain creates a downstream cost or exposes an unmet expectation

    Do not place every metric on the same level. Pick one outcome metric for the decision, the few inputs that plausibly move it, and guardrails that reveal harmful tradeoffs. For example, order completion may be the outcome, product-to-checkout conversion an upstream input, and returns and refunds the downstream guardrails.

    This hierarchy also improves your OKRs. Launching a checkout redesign is an output. Improving order completion for a defined customer segment without worsening refunds is an outcome. The second formulation gives a team room to discover the right intervention and makes success observable.

    Compare like with like before you call something a gap

    Many benchmark disagreements are really denominator disagreements. One team counts sessions while another counts people. One excludes unavailable products while another includes every product view. One reports refunds against recent orders before those orders have had time to mature. The resulting rates can look comparable while measuring different things.

    Lock the metric definition first. Then segment the result in a deliberate order:

    1. New versus returning customers. This separates the first-use experience from behavior shaped by previous purchases and existing trust.
    2. Mobile versus desktop. This exposes a device-specific journey that an aggregate conversion rate can conceal.
    3. Subscription versus one-time orders. These models represent different commitments and should not share a retention denominator.
    4. Comparable observation windows. Use the same event definitions and allow delayed outcomes such as returns, refunds, and repeat purchases to mature before comparing cohorts.

    Do not interpret the aggregate until you have inspected the segments. Overall performance can rise because the share of returning customers increased even when neither new nor returning customer conversion improved. That is a mix shift, not evidence that the product experience became better.

    For each segment, record five fields: its volume, current rate, benchmark delta, confidence in the measurement, and business exposure. Business exposure is the number of eligible journeys affected by the gap, adjusted for the value of the outcome. This prevents a dramatic percentage gap in a tiny segment from automatically outranking a modest gap in the dominant journey.

    Keep external and internal comparisons separate. An external benchmark answers whether performance looks unusual relative to a relevant peer set. An internal comparison answers where your own experience is weakest and whether it is improving. You can have a meaningful internal opportunity even when the external rate looks healthy, and you can trail a benchmark without having enough evidence to justify a particular feature.

    The output of this step should not be a league table. It should be a ranked opportunity list with explicit scope, such as new mobile shoppers dropping between product evaluation and checkout, rather than mobile conversion is below benchmark.

    Turn benchmark gaps into testable diagnoses

    A benchmark tells you where to investigate. It does not tell you why the gap exists. Treat every explanation as a hypothesis until behavioral data, customer evidence, or an experiment supports it.

    • Weak visit-to-signup: examine the promise that brought the visitor in, the value communicated on arrival, and the exact step where signup fails. Do not optimize signup if account creation is not necessary for customers to receive value.
    • Slow time-to-first-value: inspect onboarding and the sequence before the first meaningful outcome. Define first value before optimizing speed; reaching an easy but irrelevant event faster only improves the dashboard.
    • Weak product-to-checkout conversion: investigate product discovery, value communication, decision confidence, and the validity of the product-view denominator. The customer has not entered checkout yet, so a checkout redesign is not the first conclusion.
    • Weak order completion: inspect abandonment by checkout step, validation failures, transactional errors, and differences between customer segments. Here, the evidence is concentrated after purchase intent has already been expressed.
    • Weak repeat purchase or subscription retention: compare cohorts after their first transaction and first-value event. Look for a breakdown in continued value, lifecycle communication, or the experience after purchase.
    • High returns or refunds: treat them as signals that an apparent conversion win may not have produced durable value. Examine whether expectations, the delivered experience, and the reason codes align.

    At this point, classify the gap as a working diagnosis:

    • Strategy gap: the value proposition or chosen customer problem may not be strong enough. Evidence usually appears across several steps or segments rather than in one isolated interaction.
    • Execution gap: the opportunity is concentrated in a particular stage, segment, or flow that the current experience handles poorly.
    • Measurement gap: event counts do not reconcile, definitions changed, identities are duplicated, or the result moves in ways that operational records cannot explain.

    These labels are not verdicts. They determine the next evidence you need. A strategy gap calls for stronger discovery and value-proposition work. An execution gap can move into solution testing. A measurement gap requires instrumentation repair before either conclusion is trustworthy.

    Bring product, marketing, and customer experience into the diagnosis. Marketing can explain the acquisition promise and audience. Customer experience can add contact themes, return reasons, and refund context. Product can connect those signals to the instrumented journey. The shared output should be a hypothesis card containing the affected segment, observed gap, suspected mechanism, missing evidence, candidate intervention, outcome metric, and guardrail.

    This cross-functional step matters because local optimizations can move a metric while harming the journey. More aggressive messaging may increase checkout starts but also increase refunds. Removing a step may lift completion while admitting customers who never reach value. A single funnel rate cannot tell you whether the trade was worthwhile.

    Pair reliable instrumentation with disciplined experiments

    Give every metric a contract

    An analytics tool cannot rescue an ambiguous definition. Whether you use Amplitude analytics, Pendo, or another unified analytics platform, give each decision-critical metric a written contract.

    • The business question the metric answers
    • The starting and ending events
    • The numerator and denominator
    • The unit of analysis: person, session, cart, order, or subscription
    • Eligibility rules and exclusions
    • The customer and order properties used for segmentation
    • The observation window and expected reporting delay
    • The system of record used for reconciliation
    • The owner and change history of the definition

    Use event names for facts that happened, such as product viewed, checkout started, order completed, refund issued, and return completed. Store segment context as controlled properties rather than creating a different event for every device or customer type. This keeps funnel logic understandable and reduces accidental differences between reports.

    Validate the journey end to end. Confirm that an actual customer path produces the expected event sequence, that order identifiers are unique, and that completed-order and refund totals reconcile with the commerce system. Investigate discrepancies before setting a target or announcing an experiment result.

    Delayed outcomes need explicit cohort rules. A newly completed order can enter the conversion denominator immediately, but its eventual return or refund status may still be unknown. Comparing an immature cohort with a mature one understates downstream friction by construction.

    Apply privacy by design to the taxonomy. Collect only properties required for an approved decision, restrict access, define retention, and avoid placing sensitive customer information in unrestricted event properties or free-text fields. Identity-level behavioral tracking can create privacy obligations, so involve the appropriate privacy and legal owners before expanding collection.

    Predefine how an experiment will earn a decision

    Once the metric is trustworthy, turn the diagnosis into an experiment plan. Write the plan before inspecting results:

    1. State the mechanism. Explain why the proposed change should alter the observed behavior for the chosen segment.
    2. Name one primary outcome. This is the metric that determines whether the hypothesis received support.
    3. Choose guardrails. Include the nearest credible harms, such as lower order completion, weaker retention, or higher returns and refunds.
    4. Set the minimum detectable effect. This is the smallest change worth designing the test to detect, not a prediction of the result.
    5. Size the test before launch. Use the baseline, minimum detectable effect, and statistical decision rules to determine the required sample rather than stopping when the chart looks favorable.
    6. Predefine segment analysis. Name any segment that can change the decision in advance instead of repeatedly slicing the data until one view appears successful.
    7. Write the decision rule. Specify what you will ship, revise, investigate, or reject for each plausible result.

    This discipline limits p-hacking and turns an A/B test into a decision instrument. A test that has not reached the sample required by its own plan is inconclusive under that plan; it is not evidence of no effect. A result that moves the primary metric while violating a guardrail is a tradeoff to evaluate, not an uncomplicated win.

    Tie the experiment back to an outcome-based objective. Replace launch a shorter checkout with improve order completion for new mobile shoppers while protecting refund performance, validated through a predefined A/B test. The first statement rewards shipping. The second rewards solving the measured customer and business problem.

    Not every benchmark gap deserves an A/B test. Repair unreliable telemetry directly. Use customer discovery when the suspected problem is unclear. Test a product change only when you have a credible mechanism, an observable outcome, and enough eligible traffic to support the decision rule.

    Key takeaways

    • A benchmark is useful only when it is attached to a defined segment, metric contract, observation window, and product decision.
    • Map metrics across acquisition, activation, consideration, checkout, retention, economics, and downstream friction instead of optimizing one conversion rate in isolation.
    • Segment new versus returning, mobile versus desktop, and subscription versus one-time journeys before interpreting the aggregate.
    • Treat a benchmark delta as a location signal. Customer evidence and experiments must establish the mechanism behind it.
    • Rank opportunities by affected volume, business exposure, and measurement confidence, not by the largest percentage gap alone.
    • Predefine the outcome, guardrails, minimum detectable effect, sample requirement, segment analysis, and decision rule before reading experiment results.

    Open your current scorecard and choose one journey metric that is shaping the roadmap. Write its numerator, denominator, unit, customer segment, observation window, and the decision it is meant to change. If you cannot complete that sentence, instrumentation is the next product task. If you can, take the largest decision-relevant gap and turn it into a hypothesis card with a measurable outcome and guardrail.

    The goal is not to make every number resemble a peer average. It is to know which customer problem deserves attention, which intervention changed behavior, and whether the resulting growth created durable value.

    References