Tag: Amplitude analytics

  • How to Connect Voice of Customer to Behavioral Analytics

    How to Connect Voice of Customer to Behavioral Analytics

    You have interview notes, support tickets, sales objections, app reviews, and in-product feedback. Yet the roadmap discussion still comes down to which customer complained most recently or which stakeholder tells the most persuasive story.

    The way out is not another survey. Connect each voice-of-customer theme to the behavior of the people who expressed it. You can then see whether the problem changes activation, task completion, adoption, retention, or conversion; identify where the friction occurs; and decide whether the opportunity deserves roadmap space.

    Start with the decision, not the feedback backlog

    VOC becomes useful when it can change a decision. Before analyzing a theme, ask what you would do differently if the concern proved material. Would you redesign an onboarding step, improve reporting performance, simplify permissions, clarify pricing, or leave the current experience alone?

    If the answer is unclear, the theme is not ready for prioritization. It may still be worth tracking, but it should not become a roadmap item merely because it appears frequently.

    Write the theme as a behavioral hypothesis:

    Customers who encounter or mention [theme] while attempting [job] are more or less likely to [observable behavior] within [relevant window] than comparable customers who do not.

    VOC-to-behavior hypothesis template

    A useful hypothesis contains six parts:

    • Population: The users or accounts eligible to encounter the problem.
    • Job: What they were trying to accomplish, not merely the page they visited.
    • VOC theme: The friction expressed in neutral language, such as onboarding confusion or performance slowness.
    • Behavioral signal: The action or pattern you expect to observe, such as abandonment, backtracking, repeat clicks, or slow task completion.
    • Outcome: The activation, adoption, conversion, or retention metric that could move.
    • Window: The period in which that behavior and outcome are meaningful for your product.

    For example, a complaint that a flow is too complex can become a testable expectation: affected users will take longer on a step, move backward more often, depend more heavily on tooltips, or abandon the funnel at a particular screen. Those observations will not explain the customer’s motivation on their own, but they will reveal whether the stated friction has a visible behavioral footprint.

    This distinction matters. Feedback explains how customers interpret an experience. Analytics records what happened. Neither is sufficient alone. Treat the comment as a hypothesis and observable product behavior as the evidence that tests it.

    Build a shared spine between what customers say and do

    You cannot reliably connect VOC to behavior when the two systems describe customers, product areas, and outcomes differently. The work begins with a shared measurement spine: consistent identities, timestamps, product concepts, and definitions.

    Instrument the moments that represent value

    Do not begin by tracking every click. Begin with the moments that determine whether a customer reaches value:

    • The start and end points used to calculate time-to-first-value.
    • The steps and completion event in the onboarding funnel.
    • The first meaningful use of a core feature.
    • The repeated behaviors that indicate adoption rather than experimentation.
    • The conversion event that represents a real commitment.
    • The activity and return criteria used in retention analysis.

    Each event needs an explicit trigger, a user or account identity, a timestamp, and the contextual properties required for segmentation. In a business product, retain both user-level and account-level identity where your data rules permit it. A frustrated user may submit the ticket, while account retention and revenue are measured elsewhere.

    Definitions deserve the same discipline as instrumentation. If onboarding completion means reaching one screen to Product and completing a different workflow to Customer Success, the resulting cohort comparison will settle nothing. Record the definition, owner, applicable population, and known exclusions for every decision metric.

    Amplitude analytics, Pendo, or another unified analytics platform can support funnels, cohorts, and retention curves. The platform does not remove the need for a clean event taxonomy. Better charts built on inconsistent events only make the wrong conclusion look more convincing.

    Normalize VOC without stripping away its meaning

    Customer feedback arrives in incompatible forms: a support ticket describes a blocked task, a sales note records an objection, an app review compresses several problems into one comment, and an in-product response refers to the screen the customer is currently viewing. A shared theme taxonomy makes those inputs comparable.

    For each feedback record, capture the minimum fields needed to analyze it:

    • The original wording or a reference to it, so the nuance remains recoverable.
    • A neutral theme and, where necessary, a more specific subtheme.
    • The product area and job the customer was attempting.
    • The date, touchpoint, and customer or account identifier available under your privacy and data-governance rules.
    • The customer’s lifecycle stage, plan, role, or other context needed to define an eligible comparison group.
    • Whether the customer described a symptom, proposed a solution, or did both.

    That final distinction prevents a common roadmap error. A request for another button is a proposed solution. The underlying problem may be that the current action is hard to discover, too slow, or unavailable to the customer’s role. Preserve the request, but tag the friction separately. Otherwise, you will count preferred implementations rather than customer problems.

    Keep the taxonomy small enough that different people apply it consistently. Split a theme only when the distinction would produce a different cohort, root-cause investigation, or product decision. A label that never changes analysis is administrative detail, not useful structure.

    Turn each VOC theme into a fair cohort comparison

    Once the datasets share identities and definitions, build a cohort containing the users or accounts associated with a theme. Then compare that group with customers who were genuinely capable of encountering the same experience.

    Use this sequence:

    1. Define the expressed cohort. Include customers associated with the theme during a stated period. Preserve the feedback date so you can distinguish behavior before and after the comment.
    2. Define eligibility. Exclude customers who could not access the feature, workflow, plan, permission level, or product version involved.
    3. Create the comparison cohort. Use customers with a similar lifecycle stage and opportunity to perform the job, but without the same recorded theme.
    4. Align the observation window. Give both cohorts the same opportunity to complete the funnel, activate, adopt the feature, or return.
    5. Locate the behavioral difference. Compare funnel steps, task time, navigation patterns, feature adoption, conversion, and retention where each is relevant.
    6. Segment the result. Check whether the effect is concentrated by role, plan, account type, entry path, or another product-relevant dimension.
    7. Return to the qualitative evidence. Review the wording and relevant sessions around the point where behavior diverges. This is where the probable cause becomes specific enough to design against.

    The comparison group matters as much as the expressed cohort. Users who contact support are not a random sample. They may be more engaged, more experienced, more valuable, or simply more willing to report problems. A behavioral difference therefore shows an association worth investigating; it does not prove that the theme caused the outcome.

    Timing creates another trap. A customer may open a ticket because a task already failed. If you combine activity from before and after the ticket, the analysis can confuse the cause, the failure, and the attempt to recover. Anchor the timeline to the relevant exposure or task attempt, and use the feedback timestamp as context rather than automatically treating it as the beginning of the problem.

    Interpret repeated actions carefully as well. Repeat clicks can indicate an unresponsive control, uncertainty about whether a request registered, or deliberate power use. Backtracking may reflect confusion or a legitimate comparison workflow. Pair the pattern with funnel position, timing, interface state, and customer language before naming the root cause.

    Your output should be an evidence statement, not a dashboard tour. A strong statement identifies the eligible segment, the observed difference, where it appears, the outcome associated with it, and the remaining uncertainty. That is enough for a product trio to decide whether to investigate, intervene, or stop.

    Prioritize the behavioral gap and validate the fix

    Raw feedback volume is a weak prioritization rule because it has no denominator. A theme can generate many tickets because the workflow is widely used, because the problem is severe, or because the affected customers are unusually vocal. Reach, behavioral impact, and proximity to a meaningful outcome separate those possibilities.

    Build a compact opportunity case for each material theme:

    • The eligible population and the portion associated with the theme.
    • The behavior gap between the expressed and comparison cohorts.
    • The funnel, activation, adoption, conversion, or retention outcome connected to that gap.
    • The segment in which the effect is concentrated.
    • The probable root cause and the evidence supporting it.
    • The smallest intervention capable of testing that cause.
    • The primary metric, guardrails, and uncertainty that remain.

    A practical sizing model is: eligible population multiplied by the observed behavior gap multiplied by the value of recovering the affected outcome. Use a range when the inputs are uncertain. The purpose is not to manufacture a precise forecast. It is to expose whether your business case depends on broad reach, a large outcome gap, a valuable segment, or an assumption that still needs evidence.

    Do not rank opportunities by the size of the gap alone. A large drop in a low-value side path may matter less than a smaller gap immediately before activation. Conversely, a retention difference may be associated with the theme without being caused by it. Confidence intervals and explicit assumptions help keep opportunity sizing proportional to the evidence.

    When you ship, test the causal claim you actually care about. State the eligible population, intervention, primary metric, guardrails, and minimum detectable effect before looking at results. Use an A/B test when random assignment is practical. If you must rely on a staged rollout or observational comparison, label the result accordingly and keep plausible alternative explanations visible.

    Success is not a warmer survey response by itself. The behavior implicated by the original theme should move: fewer relevant drop-offs, less unnecessary backtracking, faster task completion, stronger activation, or better retention. Sentiment can confirm that the experience feels better, but the original behavioral hypothesis should still be tested.

    What a complete feedback-to-outcome loop looks like

    One reporting experience illustrates the sequence. Customers described reporting as slow. The behavioral trail contained long load times and repeated clicks on filters, which narrowed the problem beyond the broad complaint. The response combined simpler defaults, prefetching important queries, and clearer loading states. In that case, the changes reduced perceived wait time by 42% and improved day-7 retention for the affected cohorts.

    That result is a case-specific outcome, not a benchmark to paste into another business case. The transferable lesson is the chain of evidence: customer language identified the experience, behavioral data located the friction, the intervention addressed the probable mechanism, and the affected cohort supplied the right place to measure retention.

    Make this chain part of the operating cadence. Use a weekly listening review with the product trio to classify emerging themes and flag missing instrumentation. Use a monthly synthesis to join mature themes with usage data, refresh opportunity cases, and retire claims that behavior does not support. When a change ships, return to the original expressed cohort and the relevant outcome window rather than declaring success from aggregate usage.

    Key takeaways

    • Start with the roadmap decision a VOC theme could change, then express the theme as a behavioral hypothesis.
    • Give feedback and product events a shared spine: consistent identities, timestamps, product areas, jobs, and outcome definitions.
    • Compare customers who expressed a theme with customers who had the same opportunity to encounter the experience.
    • Align observation windows and lifecycle stages before interpreting funnel, activation, adoption, or retention differences.
    • Treat cohort differences as evidence of association, not automatic proof of causation.
    • Prioritize the affected population, behavior gap, outcome value, and strength of evidence rather than ticket volume alone.
    • Validate the proposed mechanism with an experiment and a predetermined minimum detectable effect whenever random assignment is practical.

    At your next listening review, choose the VOC theme consuming the most roadmap attention. Write one behavioral hypothesis, identify the eligible cohort, and compare one outcome that would make the problem worth solving. If you cannot complete that chain, the next priority is not another feature request. It is the missing identity, event, definition, or feedback tag preventing you from making the decision responsibly.

    References

  • Amplitude Browser SDK: Turn Web Vitals Into Product Decisions

    Amplitude Browser SDK: Turn Web Vitals Into Product Decisions

    You have Web Vitals in a dashboard, but the hard question is still unanswered: does a slower or less stable experience materially change activation, conversion, or retention? If your instrumentation cannot answer that, collecting more performance data will only make the dashboard busier.

    The useful setup is not simply Browser SDK plus LCP, INP, and CLS. It is a measurement system that preserves the user’s real experience, attaches enough product context to explain the result, and connects performance to an outcome your team can improve.

    Build the measurement contract before the dashboard

    Start with the decision you want to make. A good Web Vitals implementation should tell you which experience is degraded, who encounters it, whether it is associated with a meaningful product outcome, and which intervention deserves engineering time.

    I would use one normalized event, such as web_vital_observed, rather than inventing event names for every metric and route. The metric, value, page context, and audience context then become properties. That keeps the taxonomy manageable while preserving the dimensions needed for analysis.

    Retain the raw measurement

    Record LCP, INP, and CLS as distinct metric names with their raw values and units. LCP and INP are timing measures, while CLS represents visual stability, so combining their values in one aggregate would be meaningless. A separate metric-name property lets one event schema support all three without pretending that they are interchangeable.

    Do not put labels such as good, acceptable, or poor into the event name. If you want performance bands, derive them from the raw value during analysis or store the band as an additional property. Keeping the underlying value allows you to change a threshold without rewriting history.

    Add context that leads to a decision

    The minimum useful context is not the maximum available browser context. Attach only properties that help you isolate a problem or compare an outcome:

    • page_group: a stable product category such as landing page, pricing, signup, checkout, or application workspace.
    • device_class: enough detail to separate materially different experiences without creating a fragmented taxonomy.
    • geography: the approved regional level, not unnecessarily precise location data.
    • traffic_source: useful when acquisition channels land users on different page experiences.
    • user_cohort: new, returning, activated, subscribed, or another state that matters to your product.
    • experiment_variant and release_id: the connection between a performance change and the product change that may have caused it.
    • measurement_timestamp: when the experience occurred, kept separate from the time Amplitude received the event.
    • sampling_policy: whether the event came from full collection or a documented sample.

    Prefer a controlled page group over an unrestricted URL. Raw URLs can create excessive cardinality, split one product surface across many records, and expose identifiers or query-string data that should not enter analytics. Normalize the route and redact sensitive values before transmission.

    Your event contract is ready when an analyst can move from a weak metric distribution to a specific page group, audience, release, and business outcome without asking engineering to reconstruct the session.

    Protect the experience from the code measuring it

    A Browser SDK runs in the same environment whose performance you are trying to understand. That makes collection overhead part of the product decision. An analytics implementation that worsens loading or responsiveness is not merely inefficient; it contaminates its own measurement.

    Treating the Amplitude Browser SDK as a product surface leads to five practical requirements.

    1. Keep the client-side footprint and payload focused. Collect properties that support segmentation or governance, not every value the browser can expose.
    2. Make telemetry fail safely. Rendering, navigation, and interaction must continue if analytics initialization, collection, or delivery fails.
    3. Use offline queuing and retry behavior without confusing delivery time with experience time. A delayed event still belongs to the session and release in which it was measured.
    4. Sample consistently when full collection is unnecessary. A stable sampling policy is more defensible than selectively collecting only certain devices, routes, or observed performance states.
    5. Put schema validation and compatibility checks in CI/CD. Product releases should not silently rename properties, change units, or remove the context that existing dashboards depend on.

    Sampling deserves particular care. If slow sessions are more likely to be abandoned, a delivery mechanism that captures only completed journeys can underrepresent the experience you most need to see. Keep collection independent of the outcome wherever possible, document the sampling rule, and monitor coverage by page group and device class. A sample is useful only when you know what population it represents.

    Retries create a different risk: duplicate or chronologically misplaced observations. Use a stable measurement identifier when your implementation needs deduplication, and preserve the original measurement timestamp. Otherwise, a recovered connection can make an earlier performance problem appear to belong to a later release.

    Make privacy part of the event design

    Consent-aware collection, edge redaction, and regional routing should be decided before rollout. Do not send a property and hope to clean it later. Once sensitive data enters an analytics pipeline, deletion and access obligations become harder to manage across queues, retries, exports, and downstream reports.

    Review each property with a simple test: does this value materially change a product decision? If a precise URL, identifier, or location does not pass that test, replace it with a stable category or leave it out.

    Analyze distributions alongside product outcomes

    An average Web Vital hides the pattern product teams need. One page can look acceptable on average while a valuable device segment or acquisition cohort has a consistently poor experience. Start with distributions, then segment them by page group, device, geography, traffic source, and user cohort.

    Next, pair those performance distributions with funnels and cohorts. Compare activation, conversion, retention, or revenue outcomes across ranges of LCP, INP, and CLS. Keep the metrics separate, because load speed, responsiveness, and visual stability can affect different moments in a journey.

    QuestionAmplitude viewDecision it supports
    Where is the experience degraded?Metric distribution by page group and device classSelect the surface and audience to investigate
    Does the degradation matter to the product?Outcome rate across performance rangesEstimate the strength and shape of the association
    Which change caused an improvement?Experiment variant compared on both the vital and the outcomeShip, revise, or reject the intervention
    Did a release create a regression?Performance distribution trended by releaseEscalate, roll back, or investigate the affected page group

    Look for a cliff rather than assuming a smooth relationship. Conversion might remain similar across much of the distribution and then deteriorate after a particular range. That pattern gives you a more useful target than a site-wide average: move the affected population away from the range where the outcome changes.

    Do not confuse that pattern with causation. Device capability, network conditions, geography, traffic source, and user intent can affect both performance and conversion. Segmentation reduces obvious confounding, but it does not eliminate it.

    Use experiments to prove the product effect

    Once you find an important association, test an intervention. Image optimization, lazy-loading changes, and navigation changes are useful candidates because each can alter a specific part of the experience. Randomize the intervention, not the Web Vital, and measure two results together:

    • Did the treatment improve the intended LCP, INP, or CLS distribution?
    • Did the same treatment improve activation, conversion, retention, or another declared outcome?

    A treatment that improves a performance score but leaves the product outcome unchanged may still be worthwhile for experience quality or regression prevention. It should not, however, be presented as a proven growth lever. Conversely, an outcome lift without the expected Web Vital movement means your proposed mechanism was probably incomplete.

    Prioritize opportunities using four factors: the size of the affected population, the outcome gap associated with the performance range, your confidence that the relationship is actionable, and the team’s ability to change the relevant surface. This keeps a dramatic problem on a low-traffic page from automatically outranking a smaller but widespread problem in signup or checkout.

    SEO can be a compounding benefit, but it should not replace the product case. Improve the experience for real users, verify the effect on their behavior, and treat search performance as a downstream outcome rather than the sole reason to optimize a synthetic score.

    Turn the first week into an operating loop

    Start with your top three entry pages. A one-week diagnostic is a sensible time box for establishing visibility, not a promise that you will prove causality in seven days. The first goal is to expose the distribution, validate the event quality, and identify one segment worth investigating.

    1. Choose three entry pages and assign each to a stable page group.
    2. Instrument LCP, INP, and CLS with the same normalized contract.
    3. Verify coverage, missing properties, sampling behavior, timestamps, consent handling, and unexpected values before interpreting a chart.
    4. Plot each metric’s distribution by page group and device class.
    5. Overlay one outcome that occurs close enough to the experience to support a useful decision, such as signup completion or activation.
    6. Select one high-impact segment and define an intervention that could plausibly change its experience.

    Keep the first scope narrow. Adding every route, cohort, and outcome at once creates an instrumentation program before you have proven that the model produces decisions. Once the first three pages generate a credible hypothesis, extend the same event contract instead of creating a new one for every squad.

    Define ownership before the first regression

    Product should own the page groups, business outcomes, and prioritization logic. Engineering should own collection performance, delivery resilience, release metadata, and regression guardrails. Data or analytics should own schema quality, coverage checks, and the analytical definitions used in dashboards. The appropriate privacy owner should approve consent behavior, PII controls, and regional routing.

    Then define product-level service objectives for LCP, INP, and CLS by key page group. Review performance distributions beside activation and retention in QBRs, and add release guardrails so a feature cannot quietly trade away responsiveness or stability. A site-wide objective is too blunt if signup and a low-traffic support page carry different user and business consequences.

    Your instrumentation is operational when it has all of the following:

    • A versioned event contract with documented metric units and required properties.
    • Automated checks that catch schema drift during CI/CD.
    • Known coverage and sampling behavior across important page and device groups.
    • Consent, redaction, and routing rules applied before data leaves the browser.
    • A distribution view for each Core Web Vital rather than one blended score.
    • At least one product outcome connected to the performance experience.
    • A named owner and a release response for regressions.

    This is where Web Vitals stop being a periodic performance project. They become a shared decision system for product, engineering, analytics, and privacy.

    Key takeaways

    • Use one normalized Web Vitals event and preserve the raw metric value; derive performance bands without discarding the underlying measurement.
    • Attach stable page, audience, experiment, release, timestamp, and sampling context only when it supports analysis or governance.
    • Keep analytics collection lightweight, failure-tolerant, consent-aware, and protected by schema checks.
    • Analyze distributions by meaningful segments, then connect them to activation, conversion, retention, or revenue.
    • Treat correlations as hypotheses. Use an experiment to verify that a performance intervention also changes the intended product outcome.
    • Begin with three entry pages, one nearby outcome, and one actionable segment before expanding coverage.

    On your next instrumentation ticket, require three fields beyond the SDK task: the decision the data will support, the outcome it will be joined to, and the owner who will respond when it regresses. That small change turns Web Vitals collection from telemetry into product management.

    References

  • Inside the Engine Room: How I Drive Scalable Analytics APIs, Reliability, and Performance

    Inside the Engine Room: How I Drive Scalable Analytics APIs, Reliability, and Performance

    I build and scale analytics platforms with a product mindset, and the work starts with the "middleware and compute systems that power analytics at scale." In platforms like Amplitude analytics and other unified analytics platform architectures, that foundation is what makes everything else possible.

    Day to day, I oversee the "APIs behind charts, cohorts, and metrics—driving performance, reliability, and platform scalability." When those APIs are fast and resilient, every product team—from growth to customer success—can trust the insights they use to ship, learn, and iterate.

    From an engineering leadership standpoint, I partner closely with SRE to define SLOs and error budgets, wire CI/CD pipelines for safe deploys, and track DORA metrics so we improve speed without compromising quality. This combination reduces incident management toil and shortens MTTR while keeping data freshness and query latency within strict thresholds.

    From a product management leadership lens, the goal is clarity: crisp APIs, predictable contracts, and transparent stakeholder management across data, engineering, and GTM teams. That alignment empowers product teams with reliable cohorts and metrics, accelerates experimentation, and de-risks roadmaps.

    If you’re scaling analytics, invest first in the platform layer: middleware and compute, schema governance, caching strategies, and cost-aware compute. Do that well, and the visible experience—charts, cohorts, and metrics—feels effortless, even as you grow to serve billions of events with confidence.

    Inspired by this post on Amplitude – Best Practices.

    Book a consult png image
  • A Practical Measurement System for B2B Product-Led Growth

    A Practical Measurement System for B2B Product-Led Growth

    Your dashboard can show more sign-ups, more activated users, and more feature adoption while the business becomes no healthier. In B2B, that usually happens when measurement stops at individual activity and never proves that an account reached repeatable value, stayed engaged, or developed credible expansion potential.

    You don’t need a larger metric catalog. You need a connected measurement system that follows one account from eligibility to first value, repeated value, retention, and commercial impact. That system should also tell your team where the journey broke and which decision to make next.

    Measure one customer journey at two levels

    B2B products create value through people, but the commercial relationship usually exists at the account, organization, or workspace level. This creates a measurement problem: user metrics and account metrics can each look healthy while hiding a different weakness.

    Growing active-user counts may only mean that existing customers added more seats. Growing active-account counts can conceal dependence on one enthusiastic user inside each account. Measure both levels, but don’t blend them into an ambiguous active-customer number.

    If your billing or value unit isn’t an account, substitute the correct economic entity, such as a workspace or billable organization. The important rule is that every metric names the entity being counted.

    Before building a dashboard, write a metric contract for every top-line measure. It should specify:

    • The business question and decision the metric supports.
    • The entity being counted: user, account, workspace, or revenue.
    • The qualifying population and the moment an entity becomes eligible.
    • The event or event sequence that constitutes success.
    • The observation window and the period allowed for success.
    • Exclusions for employee activity, test accounts, duplicate identities, and unusable telemetry.
    • The segments that must remain available for diagnosis.
    • The owner responsible for resolving definition or data-quality problems.

    This contract prevents a common denominator error. Invited members may create new user registrations, but they aren’t necessarily new accounts. If they enter the activation denominator as though they started a new buying journey, the rate stops answering a coherent question.

    Your event model must also resolve each action to the account or workspace in which it occurred. Assigning an event to a user’s current account can corrupt historical reporting when that user belongs to multiple workspaces or changes organizations.

    Decision questionPrimary unitUseful measuresWhat a weakness helps you locate
    Did a new account reach meaningful value?Account or workspaceActivation rate and time-to-first-valueAcquisition quality, setup friction, or an unclear value path
    Is value becoming repeatable?Account and user roleRecurrence of the core behavior, active accounts, and feature adoptionShallow adoption, novelty effects, or dependence on one champion
    Does usage endure?Account cohortCohort-based product retentionA gap between initial success and durable value
    Is product value creating commercial pull?Account and revenueValidated expansion intent plus expansion and contraction revenueA weak commercial signal, packaging mismatch, or failed handoff
    Can the experience scale responsibly?Account and operationsSupport deflection, incident signals, and delivery guardrailsGrowth that is shifting cost or reliability problems elsewhere

    A useful portfolio view therefore combines activation, onboarding completion, time-to-first-value, active accounts, feature adoption, cohort retention, expansion and contraction revenue, and support deflection. These aren’t interchangeable scorecard tiles. Each one answers a different question in the value chain.

    Treat activation as a hypothesis about future retention

    Activation isn’t whatever happens at the end of your onboarding checklist. It is your current hypothesis about the earliest observable behavior that shows a qualified account has received meaningful product value.

    That distinction matters. In a hypothetical collaboration product, inviting a colleague may be necessary setup. Completing a shared workflow may be the first evidence of value. Calling the invitation activation would reward the team for moving people through configuration, even if the product never solves the underlying job.

    A credible activation definition should meet several tests:

    • It represents delivered value, not mere exposure to a screen or feature.
    • It occurs early enough for product, marketing, and customer-success teams to influence it.
    • It can be measured consistently for the eligible population.
    • It respects different use cases when those use cases have materially different value paths.
    • It is associated with stronger later retention inside comparable cohorts and segments.

    The last test is important, but it doesn’t establish causality. Accounts that activate may already have greater intent, better internal sponsorship, or a more suitable use case. Treat the relationship as evidence that improves your hypothesis, then use controlled interventions where practical to learn whether removing a particular barrier changes downstream behavior.

    Use the same contract to define the related measures. Activation rate is the share of eligible accounts completing the activation behavior within the agreed window. Time-to-first-value begins at the same eligibility moment and ends at the same success event. Onboarding completion remains a diagnostic measure unless completing onboarding itself delivers the promised outcome.

    A practical validation loop looks like this:

    1. Map the path from eligibility through setup to the proposed first-value event.
    2. Use funnels and segmentation to locate the step where qualified accounts stop progressing.
    3. Compare later retention for accounts that did and didn’t complete the candidate behavior within equivalent use-case, acquisition, and account cohorts.
    4. Inspect the time-to-first-value distribution by segment instead of relying on one blended average.
    5. Test a focused intervention at the identified bottleneck, such as simpler setup, clearer messaging, a contextual guide, or a revised product tour.
    6. After any activation lift, check repeated use and cohort retention before declaring that the growth system improved.

    This is where funnels, high-signal behavioral segments, retention cohorts, and A/B tests on messaging or in-app guidance belong in the same workflow. The funnel identifies friction. The cohort tests whether the behavior matters. The experiment tests whether your intervention changes it.

    If activation rises while later retention stays flat, don’t celebrate the dashboard. Either the activation behavior is too shallow, the experiment generated temporary compliance, or the product fails to deliver enough value after the first success. Each explanation produces a different roadmap decision.

    Use a driver tree to show exactly where growth breaks

    A flat scorecard tells you what changed. A driver tree shows where to investigate. For many B2B PLG products, the measurement chain can be expressed as:

    Eligible accounts → setup complete → activated → repeated core value → retained active accounts → expansion intent → expansion or contraction revenue.

    This isn’t a universal linear funnel. Renewal and expansion can overlap with ongoing adoption, and different roles may enter at different points. Its purpose is to expose the assumptions connecting product behavior to business performance.

    Read movement between adjacent stages before reaching for a broad explanation:

    • If eligible accounts grow while activation falls, split acquisition quality from product friction. Compare equivalent acquisition and use-case segments before changing onboarding.
    • If onboarding completion improves while activation doesn’t, you probably removed checklist friction without improving the first-value experience.
    • If activation improves while repeated value doesn’t, inspect whether the activation event is too shallow or the initial experience creates novelty rather than a durable habit.
    • If active users increase while active accounts remain flat, adoption may be deepening inside existing customers without broadening the account base.
    • If repeated product value is healthy while account retention or revenue weakens, product telemetry alone can’t explain the result. Join account behavior with customer status and commercial data.
    • If expansion-intent signals rise while expansion revenue stays flat, validate the signal and inspect the go-to-market handoff before assuming the product created qualified demand.

    These patterns narrow the search; they don’t prove a cause. A driver tree should help your team decide which segment, journey step, qualitative evidence, or experiment to inspect next.

    The same tree separates leading indicators from lagging outcomes. Setup completion and high-signal power-user actions can lead into active usage and cohort retention, while expansion and contraction revenue arrive later. A leading metric earns its place only when you continue testing its relationship with the outcome it is supposed to predict.

    This changes how you write product OKRs. “Launch a new onboarding tour” is an output. “Increase validated activation for qualified accounts without weakening downstream retention or support outcomes” is an outcome. The first statement rewards shipping. The second forces the team to state the behavior it expects to change and the evidence required to keep investing.

    For every experiment, record the target segment, affected driver, hypothesis, exposure event, primary outcome, guardrails, analysis window, and downstream validation. Don’t call a variant successful because it increased tutorial clicks when the intended outcome was account activation.

    Keep operational guardrails beside growth outcomes. Incident management and DORA measures can complement product metrics when faster experimentation or adoption adds reliability risk. Support deflection provides another check: apparent growth is less attractive if it merely transfers unresolved friction to customer support.

    Segment for decisions, then use benchmarks for calibration

    A blended retention curve is an average of customers who may have different jobs, expectations, acquisition paths, and product cadences. It can improve because your customer mix changed even when no segment received a better experience.

    Build cohorts from a consistent starting event, such as the moment an account becomes eligible to pursue first value. Then define retention using a value-bearing behavior appropriate to the product’s natural cadence. A product used for an occasional but critical workflow shouldn’t be forced into a weekly-use definition merely because weekly activity is easy to chart.

    Keep three concepts separate:

    • User retention asks whether a person or role continues using the product.
    • Product-level account retention asks whether the original account cohort continues completing the qualifying value behavior.
    • Commercial retention asks what happened to the cohort’s revenue after expansion and contraction.

    One cannot substitute for another. An account may retain its contract while meaningful product use declines. Another may show healthy usage while commercial contraction occurs. That gap is information, not an inconvenience to smooth out.

    Start with segments that can change an actual decision:

    • Primary use case or job-to-be-done, when value paths differ.
    • Self-serve versus sales-assisted acquisition, when expectations or onboarding support differ.
    • Account size or plan, when collaboration depth and feature access differ.
    • Administrator, champion, and end-user roles, when each role contributes differently to value.
    • New versus established accounts, when the same behavior means something different at each lifecycle stage.

    Resist slicing until every cell becomes noisy. A useful test is simple: if a segment underperforms, would you choose a different intervention or owner? If not, it probably doesn’t belong on the operating dashboard.

    Treat expansion intent with the same discipline as activation. Seat invitations, adoption of a higher-value workflow, or repeated encounters with a product limit may be plausible candidates, but none should be accepted on intuition alone. Compare each signal with later expansion outcomes by account segment. Keep the label “intent” until the behavior proves commercially predictive.

    Sales involvement doesn’t invalidate product-led measurement. Keep a shared lifecycle definition, segment the acquisition or expansion motion, and distinguish product-sourced, product-assisted, and merely product-active accounts using explicit attribution rules. Otherwise, any active customer can be retroactively called product-led.

    External benchmarks are most useful after your internal definitions are stable. Before comparing rates, verify the unit of analysis, eligibility rule, event semantics, observation window, segment mix, and treatment of assisted accounts. Peer-informed targets can calibrate ambition and help identify gaps, but a benchmark built from a different denominator is not a target. It is a false comparison.

    Your operating view should ultimately answer four questions without a forensic exercise: Which segment moved? At which stage? Did a downstream outcome confirm the movement? What decision changes because of it? If a metric can’t help answer one of those questions, it belongs in a diagnostic workspace rather than the executive scorecard.

    B2B PLG measurement FAQ

    Should the headline metric count users or accounts?

    Use the economic value unit for the headline and user-level measures for diagnosis. For most B2B products, that means retained active accounts or workspaces completing a validated value behavior. Role-based user measures then reveal whether adoption is broad, concentrated in a champion, or blocked for a critical participant.

    What is the best north-star metric for B2B product-led growth?

    There is no context-free north-star metric. Choose a value-bearing account behavior that naturally recurs and has a defensible relationship with retention. Pair it with activation, expansion and contraction, and reliability guardrails so one optimized number cannot hide damage elsewhere.

    Should sales-assisted accounts be excluded?

    No. Excluding them can remove a material part of the customer journey and overstate the independence of the product motion. Keep the lifecycle and value definitions consistent, label the acquisition or expansion path, and compare segments. Product-led growth doesn’t require sales-free growth; it requires clarity about what product behavior contributed.

    When should the activation definition change?

    Change it when the product’s value proposition, target job, telemetry, or evidence linking activation with retention materially changes. Version the definition and avoid splicing incompatible measures into one time series. Backfill the new definition only when the historical event data supports it; otherwise, mark a clean break.

    Before your next roadmap review, write the metric contracts for one activation behavior and one retained-account behavior. Connect them to expansion and contraction, add a reliability or support guardrail, and ask every major roadmap bet to name the link it should move. If a bet can’t state its expected behavioral outcome and downstream confirmation, you have found a strategy gap before spending the engineering effort.

    References

  • Unlock Real-Time Product Insights: Amplitude + OpenAI MCP in ChatGPT, Without BI Bottlenecks

    Unlock Real-Time Product Insights: Amplitude + OpenAI MCP in ChatGPT, Without BI Bottlenecks

    I’ve been working to remove the friction between product questions and product answers. The most impactful step so far: connecting Amplitude analytics directly into ChatGPT via OpenAI’s MCP. This turns everyday conversations into decision-grade insights—no dashboards to hunt, no SQL to write, and no analytics queue to wait on.

    Connect Amplitude data directly to the tools your team uses every day. OpenAI’s MCP connector eliminates traditional barriers to product data.

    In practice, this means I can ask ChatGPT natural-language questions like, “Where are users dropping in our activation funnel this week?” or “Which cohorts are driving retention lift post-onboarding?” and get grounded answers from Amplitude—fast. It’s a step-change for product-led growth because the insights live where we already think and plan.

    Here’s how I apply it day to day: I’ll prompt ChatGPT to compare week-over-week activation for new SMB signups across regions, diagnose drop-offs by step, and summarize A/B testing outcomes with guardrails like minimum detectable effect considerations. When we’re shaping strategy, I’ll pull a retention analysis and cohort breakdown to inform bet sizing and roadmap tradeoffs—all without pulling the team into a BI bottleneck.

    Governance remains non-negotiable. I scope the MCP tools to a least-privilege data slice, apply privacy-by-design rules to exclude PII, and log every query for auditability. Clear data governance and AI risk management policies ensure we maintain trust while accelerating discovery. Tight context window management keeps prompts focused and reduces noise.

    Operationally, the setup is straightforward: define the MCP tool spec for Amplitude, map canonical events and metrics (activation, retention, conversion, and product-qualified lead stages), and test with a retrieval-first pipeline so responses reliably cite the right source of truth. We standardize metric definitions across product, growth, and customer success to avoid semantic drift.

    The impact on empowered product teams is immediate. Continuous discovery becomes a daily habit rather than a quarterly ritual; questions move from “I’ll get back to you” to “Let’s check right now.” For product managers working with LLMs, this is the connective tissue that makes ChatGPT a true ChatGPT connector for analytics—an on-demand, unified analytics platform that supports faster iteration and sharper decision-making.

    If you’ve been waiting to make analytics truly ambient, this is the moment. Start small with a single funnel or cohort, validate governance, and expand to your core lifecycle metrics. The payoff is a shared understanding of what’s working, what’s not, and where to focus next—delivered in the flow of work.

    Inspired by this post on Amplitude – Best Practices.

    Book a consult png image
  • Product Analytics for Everyone: Master Funnels, Retention, and Conversion to Drive Growth

    Product Analytics for Everyone: Master Funnels, Retention, and Conversion to Drive Growth

    Product analytics isn’t a specialist’s sport—it’s a team capability. In my role leading product teams, I’ve seen designers, engineers, marketers, and customer success partners uncover insights that shape strategy, accelerate product-led growth, and improve outcomes for customers. When we demystify the basics and bring analytics into everyday decisions, we build truly empowered product teams.

    Here’s the core promise of this approach: "Learn the product analytics fundamentals of funnels, retention, and conversion drivers so that anyone can confidently answer key product questions." That line has guided how I teach product managers to think—start with the essentials, tie them to real customer behaviors, and make the work repeatable across the organization.

    I start with funnels because they tell a story—the journey from discovery to value. A simple example: track the path from sign-up to user activation to the first value event. This reveals where onboarding succeeds or stalls, what friction blocks adoption, and which moments are ripe for optimization. With tools like Amplitude analytics or Pendo, we can break down conversions by segment, channel, or feature usage to isolate where improvements matter most.

    Next comes retention analysis, the clearest signal that we’re building something customers choose to return to. Cohort analysis shows who comes back and when; retention curves show where value compels a second, third, and tenth use. Tie retention to activation milestones and the outcomes customers achieve—not just logins—and you’ll quickly spot whether your product discovery assumptions hold up in the wild. A unified analytics platform makes these insights discoverable and repeatable across teams.

    Conversion drivers round out the picture. Once the funnel is clear and retention is stable, I look for the behaviors and experiences that predict success: feature combinations, time-to-value, message timing, or supportive content. Whether in Amplitude analytics or Pendo, correlating these drivers with outcomes lets us prioritize roadmaps with confidence. Pair this with continuous discovery—qualitative interviews, in-product feedback, and rapid experiments—and you’ll move from interesting data to decisive actions.

    This is how we build empowered product teams: by making analytics a daily habit rather than a quarterly report. We bring insights into roadmap reviews, design critiques, and sprint planning; we celebrate learning from experiments as much as shipping features; and we hold ourselves accountable to customer outcomes, not just output. When everyone can interpret funnels, discuss retention, and isolate conversion drivers, we make smarter bets faster.

    If you’re getting started, keep it simple. Define a clear activation metric, instrument the top of your funnel, and track a small number of cohorts. Share a weekly readout with highlights, surprises, and questions to investigate. Over time, stitch insights into narratives that drive product-led growth—and, most importantly, help customers achieve what they came for.

    Product analytics isn’t just for analysts. It’s a shared language for product discovery, onboarding excellence, user activation, and long-term retention. When we practice it together, we build better products and stronger teams.

    Inspired by this post on Amplitude – Best Practices.

    Book a consult png image
  • How Amplitude AI Feedback Turns Noise into Product Signal You Can Ship With Confidence

    How Amplitude AI Feedback Turns Noise into Product Signal You Can Ship With Confidence

    I’ve spent enough time in the trenches of product management to know the hardest part isn’t collecting feedback—it’s separating signal from noise. When every channel is buzzing, the real question becomes: what should we build next, and why? That’s where Amplitude AI Feedback has changed how I work. It gives me a disciplined, data-informed way to turn messy qualitative input into clear, defensible roadmap decisions.

    Learn how Amplitude AI Feedback leverages AI to transform massive volumes of customer feedback into actionable product insights.

    In practice, this means I can synthesize input from support tickets, NPS responses, user interviews, sales notes, and reviews—then connect those insights to product behavior data from Amplitude analytics. The result isn’t just a list of requests; it’s a ranked problem set grounded in evidence, which makes product discovery and continuous discovery faster, clearer, and less biased.

    A recent example: we were hearing recurring complaints about onboarding friction, but it wasn’t obvious which steps truly mattered. By pairing feedback themes with activation and retention signals, I could zero in on the first-session setup tasks that correlated with drop-off. That clarity guided product roadmapping and sprint planning decisions we could stand behind, and it accelerated user activation without bloating the backlog.

    My workflow is straightforward: aggregate feedback, cluster themes, validate with behavioral metrics, and translate insight into outcomes. I look for patterns tied to user activation, retention analysis, and moments that drive product-led growth. When the evidence shows a request is both frequent and high-impact, it earns a place on the roadmap; when it’s loud but low-impact, it becomes a targeted experiment rather than a default commitment.

    What I appreciate most is the confidence this brings to stakeholder conversations. Instead of debating opinions, we review the evidence: quantified themes, clear user stories, and measurable KPIs. That turns “Finally, Signal That Tells You What to Build” from a slogan into an operating principle, and it helps empowered product teams move faster with fewer reversals.

    If you’re building your AI Strategy or exploring LLMs for product managers, this is one of the highest-leverage moves you can make: use a unified analytics platform to connect qualitative feedback with quantitative behavior. It sharpens prioritization, improves time-to-learning, and keeps the team focused on outcomes—not outputs.

    Inspired by this post on Amplitude – Best Practices.

    Book a consult png image
  • A Quality System for Trustworthy AI-Assisted UX Research

    A Quality System for Trustworthy AI-Assisted UX Research

    Your AI-generated synthesis can be polished, plausible, and wrong. The dangerous failures are rarely obvious fabrications. They are quieter: a biased sample becomes a universal claim, a participant’s opinion becomes a product need, or a tidy theme loses the contradiction that should have changed the roadmap.

    If you are deciding whether to trust AI-assisted UX research, do not judge the fluency of the summary. Judge the evidence chain behind it. You need to see how a product decision connects to the participants recruited, the questions asked, the underlying observations, the analytical interpretation, and the behavioral data used to check it.

    Key takeaways

    • Research quality is mostly determined before an AI tool sees a transcript. Start with the decision, learning question, and hypothesis.
    • Use AI to accelerate transcription, extraction, tagging, clustering, and contradiction searches. Keep interpretation, confidence, and product judgment under human control.
    • Require every theme to retain its participant coverage, supporting evidence, counterexamples, and unresolved uncertainty.
    • Pair qualitative findings with funnels, cohorts, session evidence, and CRM data when those signals are relevant. Neither qualitative nor quantitative evidence should carry the decision alone.
    • Finish with an atomic insight and a recorded choice. A summary that does not change a decision, test, or learning priority is not finished research.

    Define quality at the decision boundary

    Many teams begin AI-assisted research by asking which model should summarize their transcripts. That is too late in the process. The first quality control is the decision the research must inform.

    Strong discovery begins with a decision statement, an explicit learning goal, and a hypothesis the team is willing to falsify. Without those constraints, an AI system can generate an impressive taxonomy of themes while leaving the actual product question untouched.

    Before recruiting participants or writing prompts, create a short research contract:

    • Decision: Name the choice that is genuinely open. Examples include whether to pursue an opportunity, which problem to solve first, or whether a proposed workflow deserves further testing.
    • Decision condition: State what you would need to learn to proceed, pause, narrow the audience, or reject the current direction.
    • Learning question: Ask about the behavior, context, constraint, or unmet need that makes the decision uncertain.
    • Hypothesis: Write the current belief in a form that evidence could disprove. If every possible interview result would support it, it is not a useful hypothesis.
    • Relevant population: Specify whose behavior matters to this decision and which segments could experience the problem differently.
    • Evidence plan: Identify what interviews can reveal and which behavioral or operational signals could challenge the interpretation.
    • Data boundary: Decide what the AI tool is allowed to receive, what must be removed, and who may review the resulting artifacts.

    This contract changes how you evaluate the output. You are no longer asking whether the summary sounds reasonable. You are asking whether the evidence changes a named choice under stated conditions.

    My standard is simple: a decision-grade insight must survive a skeptical review without relying on the model’s authority. A reviewer should be able to inspect the underlying evidence, see which participants and segments it covers, understand the interpretation applied to it, and identify what remains unknown.

    Keep one distinction visible throughout the work:

    • Observation: What the participant did, described, showed, or failed to complete.
    • Interpretation: What that behavior may mean about a goal, anxiety, constraint, or job.
    • Implication: What the product team may choose to change, test, or leave alone.

    AI can help produce all three, but it should never blur them into a single sentence. Once an inference is written as if it were an observed fact, the rest of the synthesis becomes difficult to audit.

    Protect the signal before AI touches it

    An LLM cannot repair a convenient sample or a leading interview guide. It can only reorganize the resulting bias, often in language that makes the bias look more certain.

    Recruit for the decision, not for convenience

    If you interview only power users, you risk treating advanced workflows as mainstream needs. If you interview only vocal detractors, the roadmap can become a queue of complaints. A more useful recruiting frame includes new users, churned users, people who evaluated but did not convert, and adjacent personas where the decision calls for them.

    Build a participant matrix before outreach. Use rows for the segments that could materially change the decision and columns for relevant states, such as adoption stage, conversion outcome, or workflow maturity. The matrix is not a quota formula. It is a visibility tool. It should make overrepresented groups and missing perspectives obvious.

    Carry that segment metadata into synthesis. A theme that appears among established customers should not silently become a claim about evaluators. When a segment is absent, write that limitation into the insight rather than hiding it in an appendix.

    Ask for behavior before interpretation

    Questions about whether someone likes an idea invite speculation, politeness, and solution theater. Ask about the last relevant event instead. Have the participant reconstruct what triggered it, what they tried, where they hesitated, who else became involved, what workaround they used, and what happened next.

    Neutral, behavior-first questions become stronger when participants can support the account with artifacts such as screenshots or workflow examples. The artifact does not automatically prove the interpretation, but it helps distinguish remembered behavior from a general opinion.

    Pilot the guide with the product trio. Remove product terminology that telegraphs the preferred answer. Check whether each question could produce evidence against the working hypothesis. If the guide repeatedly asks participants to react to your solution, it is a concept evaluation guide, not an open discovery guide. Label it accordingly.

    Set privacy boundaries before uploading transcripts

    Consent to an interview does not automatically settle how AI will be used in transcription, analysis, storage, or sharing. Tell participants how their material will be handled, follow your organization’s data governance requirements, and remove identifiers that are not needed for the decision.

    Do not place sensitive participant data into an unapproved prompt workflow. If the tool’s handling, retention, or access controls have not been approved, keep raw transcripts out of it and work with appropriately de-identified material in an authorized environment. The downside is not merely a poor synthesis; it is unnecessary exposure of participant and customer information.

    De-identification should not erase the context required for analysis. Preserve non-identifying segment labels, workflow stage, and participant codes when they are relevant. The goal is to minimize sensitive data while retaining enough context to audit coverage and interpretation.

    Make AI produce an auditable synthesis

    The most reliable workflow separates extraction from clustering and clustering from judgment. Asking for findings, recommendations, sentiment, and a roadmap in one prompt encourages the model to fill gaps and compress uncertainty.

    1. Prepare the evidence set. Preserve the original transcript or recording, assign a participant code, attach relevant segment metadata, and remove unnecessary identifiers. Do not let an AI-generated summary replace the underlying material.
    2. Extract participant-level observations. Ask the model to work through each participant separately. Capture the behavior or event, its context, the supporting excerpt or evidence location, and any missing information. Do not ask for themes yet.
    3. Review the extraction. Check whether the observation is grounded in the transcript and whether the model has converted an opinion into behavior or inferred a motive the participant did not provide.
    4. Cluster reviewed observations. Group similar evidence only after the participant-level pass. Require each cluster to retain the contributing participant codes, segment coverage, supporting evidence, and meaningful variations.
    5. Search for contradictions. Ask which observations do not fit the cluster, which participants experienced the situation differently, and which alternative explanations remain plausible. Do not treat dissent as noise merely because it makes the summary less tidy.
    6. Draft atomic insights. Turn a defensible pattern into a small evidence packet containing the finding, evidence, coverage, contradictions, confidence rationale, product implication, and unresolved question.
    7. Triangulate relevant claims. Compare the qualitative interpretation with funnels, cohorts, session evidence, in-product paths, or CRM data when those systems contain a useful signal.
    8. Conduct the decision review. A person accountable for the product choice inspects the evidence chain, challenges the interpretation, and records what the team will do or learn next.

    You can make the separation explicit with narrowly scoped prompts.

    Extraction prompt: Use only the supplied transcript. For each relevant event, return the participant code, observed or reported behavior, context, supporting excerpt, evidence location, and uncertainty. Do not merge participants, infer motives, or recommend a solution. Flag information that is missing.

    Clustering prompt: Use only the reviewed observations. Group evidence by shared behavior and context. For every cluster, retain participant codes, represented segments, supporting observations, material variations, counterexamples, and plausible alternative explanations. Do not use repetition in the transcript as a substitute for participant coverage.

    Challenge prompt: Review the proposed themes as a skeptical researcher. Identify unsupported generalizations, segment differences that were flattened, interpretations written as observations, contradictory evidence, and claims that cannot be traced to the supplied material. Do not invent missing evidence.

    Prompt design helps, but it does not replace review. Keep the prompt, relevant tool or model information, input scope, and human corrections with the research artifact. If the synthesis later changes, you should be able to determine whether the cause was new evidence, a different analytical instruction, or a human judgment.

    AI is well suited to accelerating transcription, tagging, theme clustering, Jobs to Be Done extraction, and searches for hesitation or sentiment. Treat the latter outputs as interpretations to validate, not measurements generated by an objective instrument. A sentiment label is useful only when a reviewer can return to the behavior and language that produced it.

    Validate the insight, then record the decision

    A good synthesis review is not a copy-edit. It is an attempt to break the claim before the claim influences a roadmap.

    Run a quality review against the evidence chain

    • Traceability: Can a reviewer move from the insight to the contributing participants and the exact supporting material?
    • Coverage: Does the claim name the segments represented, and does it disclose relevant segments that are missing?
    • Construct validity: Is the finding about the behavior the study intended to understand, or has a nearby opinion been used as a proxy?
    • Separation: Are observation, interpretation, and product implication visibly distinct?
    • Contradiction: Does the artifact preserve disconfirming cases and material variations instead of forcing consensus?
    • Triangulation: Where behavioral data is relevant, does it support, narrow, or challenge the qualitative account?
    • Decision relevance: Does the finding change a live choice, a test, or the next learning priority?

    Do not outsource confidence to the model. A confident tone is a language property, not an evidence assessment. Record confidence as a human rationale based on the clarity of the underlying behavior, the relevance and coverage of participants, consistency and counterexamples, and any corroborating behavioral evidence.

    Quantitative and qualitative signals answer different parts of the question. Funnels, cohorts, and retention analysis can show where behavior changes or where people leave. Interviews and artifacts can expose the goals, anxieties, organizational constraints, and workarounds behind that behavior. Pairing those signals is how a team moves from observing what happened to developing a testable account of why.

    When the signals disagree, do not average them into a vague conclusion. Check whether the interview sample represents the population in the analytics, whether the event instrumentation reflects the behavior being discussed, whether segments have been combined, and whether the evidence refers to the same stage of the journey. A contradiction is often the next research question.

    Use an atomic insight format

    A reusable insight should be small enough to inspect and complete enough to guide a choice. Use this structure:

    • Decision: The product choice this evidence informs.
    • Finding: The observed behavioral pattern and the context in which it occurs.
    • Evidence: Participant codes, excerpts or artifact locations, and any relevant behavioral signal.
    • Coverage: The represented segments and known gaps.
    • Interpretation: The best current explanation, clearly labeled as an inference.
    • Contradictions: Cases or data that weaken, narrow, or complicate the interpretation.
    • Confidence: A short rationale grounded in evidence quality, coverage, consistency, and triangulation.
    • Product implication: The opportunity, risk, constraint, or tradeoff the team should consider.
    • Disposition: Act, test further, monitor, or take no action.
    • Next unknown: The uncertainty most likely to change the decision.

    Useful insight records also prevent familiar synthesis mistakes. Replace a broad label such as onboarding friction with the specific behavior, actor, context, and consequence. Do not let a memorable quotation stand in for a pattern. Do not describe a participant’s requested feature as the underlying need. Do not convert an AI-generated cluster into a roadmap item until the evidence packet survives review.

    Bring the atomic insights to a decision review with the product trio. Record the choice, its rationale, what the team is deliberately not doing, and the evidence that could reopen the decision. Connect the chosen action to an outcome or learning objective rather than treating delivery of a feature as proof that the research was correct.

    For your next study, start with one live decision and run the evidence through this chain. If a theme cannot be traced, mark it as a hypothesis. If participant coverage is lopsided, narrow the claim. If qualitative and behavioral evidence conflict, investigate the conflict before committing the roadmap. That is how AI becomes a fast, inspectable research assistant instead of an unaccountable author of customer truth.

    References

  • From Sketch to Clickable Demo: My AI Prototyping Playbook to Build Apps in Hours

    From Sketch to Clickable Demo: My AI Prototyping Playbook to Build Apps in Hours

    I’ve spent much of my career compressing the distance between a napkin sketch and something real customers can touch. At HighLevel, my product teams use generative AI to validate ideas faster, reduce risk earlier, and win stakeholder trust with evidence instead of slides. The goal isn’t to be flashy—it’s to be precise, testable, and repeatable.

    Today, you can build it before you pitch it. AI prototyping can turn ideas into clickable demos in hours. Here are some tools to try and steps to follow.

    I start every AI prototyping sprint by sharpening the problem statement and the outcome we care about. That means being explicit about the target user, jobs-to-be-done, and the riskiest assumptions. I define a minimum detectable effect (MDE) and tie it to outcomes vs output OKRs so everyone aligns on what “good” looks like before we touch a tool.

    From there, I move from sketch to interface. I capture a rough flow (whiteboard, tablet, or even paper) and generate UI variations with my AI product toolbox—tools that translate structure into components and screens. I’ll iterate on information hierarchy and copy until the narrative supports the core job, borrowing techniques from UX writing. For product managers leaning into LLMs for product managers, this phase is about speed to feedback, not perfection.

    Next, I wire data and logic. I connect a lightweight backend or spreadsheet, stitch in a CRM integration if needed, and add LLM calls through a ChatGPT connector or Claude Code. If the concept benefits from multi-step autonomy, I introduce agentic AI to orchestrate tasks across APIs. CustomGPT workflows help me encapsulate business rules so the demo behaves consistently in user paths we care about.

    Governance is not optional at this stage. I apply privacy-by-design defaults, document data governance decisions, and run a quick AI risk management pass: input validation, prompt safety, rate limits, and fallback responses. This keeps the prototype credible and prevents false positives from polluting stakeholder perception.

    With a click-through in hand, I instrument the experience so learning compounds. I drop in Amplitude analytics to track activation, task completion, and drop-off, and set up simple A/B testing when there’s a meaningful design or copy choice. This makes the prototype a learning vehicle, not just a demo.

    Then I get it in front of users—fast. Five targeted conversations will beat fifty internal opinions. I run structured product discovery interviews, observe time-to-value, and capture objections. This is where empowered product teams shine: we make changes in real time, re-run the flow, and document what moves the needle for product-led growth.

    When speed matters, I use a four-hour cadence: Hour 1 for problem framing and MDE; Hour 2 for sketch-to-UI generation; Hour 3 for data wiring and AI logic; Hour 4 for instrumentation and user walkthroughs. By the end, we have a clickable demo, preliminary analytics, and a clear decision on whether to advance, pivot, or park.

    Finally, I translate insights into a concise artifact: the hypothesis we tested, the signal we observed, the trade-offs we made, and the next sprint plan for product roadmapping and sprint planning. The point is not to be right on the first try; it’s to learn precisely, cheaply, and quickly enough to invest with conviction.

    If you adopt this approach, you’ll find that stakeholder management becomes easier, team energy rises, and your roadmap earns credibility. Build it before you pitch it, and let real interactions—not wishful thinking—do the heavy lifting.

    Inspired by this post on Product School.

    Book a consult png image
  • Cut Time to Value, Boost Retention: My Proven Playbook for Activation, Growth, and Loyalty

    Cut Time to Value, Boost Retention: My Proven Playbook for Activation, Growth, and Loyalty

    Time to value is the most reliable early indicator of long-term user retention I know. When customers experience meaningful product impact fast, they stick around, expand, advocate, and cost less to support. Over the years leading product teams, I’ve learned that speed-to-impact isn’t a nice-to-have—it’s the engine behind sustainable product-led growth and efficient go-to-market.

    Accelerate retention by reducing time to value. Learn how faster product impact drives growth, reduces costs, and keeps users engaged in the long term.

    Practically, I define time to value as the duration from first touch (or first login) to the moment a user achieves their “aha” outcome—something tangibly useful aligned to their job-to-be-done. The shorter that journey, the higher the likelihood of user activation, trial conversion, and durable engagement. This is why I obsess over onboarding, in-app guides, product tours, and the clarity of our value proposition.

    My first move is to map the Minimum Path to Value (MPV): the smallest set of actions needed to deliver a real result for a new user. I strip away everything non-essential in that path—fields, clicks, choices, and jargon. Opinionated defaults, smart templates, sample data, and single-player workflows let customers succeed in minutes, not days. The goal is to reduce cognitive load while making the next best action unmistakably clear.

    Instrumentation turns TTV from a hunch into a system. I track activation events, cohort retention, and conversion using platforms like Amplitude analytics and Pendo, with timely nudges through Intercom when users stall. I look at the distribution of TTV (not just the average), correlate it with retention analysis, and set explicit targets such as “new users reach first value within 10 minutes.” Those targets become team-level outcomes—not outputs—and we review them weekly.

    Experimentation is how we iterate toward the fastest path to value. I rely on A/B testing to compare onboarding flows, progressive profiling to delay non-critical inputs, and opinionated setup wizards to remove guesswork. Auto-generated example projects, pre-configured integrations, and guided checklists accelerate user activation without sacrificing flexibility for advanced users.

    Content and guidance matter as much as UX. Tooltips, contextual in-app guides, and short product tours should be timely, skippable, and laser-focused on the outcome, not the feature. I pair these with a concise knowledge base and short explainer videos that reinforce the same value narrative a user sees inside the product.

    Cross-functional alignment is essential. Product, marketing, sales, and customer success must rally around the same activation metric and TTV target. That alignment ensures our trial messaging, onboarding emails, and CS playbooks don’t compete—they compound. When everyone points to the same first-value moment, friction drops and adoption rises.

    Pricing and packaging can also accelerate time to value. Free trials should be long enough for users to credibly reach first value; usage-based gates should never block the MPV. I prefer to unlock everything needed to hit the “aha” moment, then meter after the value is viscerally felt—this respects the user’s time and reinforces trust.

    There’s a cost story, too. Faster time to value reduces tickets, shortens onboarding cycles, and lowers cost-to-serve. It also clarifies product discovery: when we see where users stall, we don’t guess at roadmap priorities—we let the data guide our next bet.

    In my experience at HighLevel, I’ve repeatedly seen activation rates jump when we cut time to value from days to minutes. The specific tactics vary by product, but the pattern holds: when the first outcome is undeniable and fast, retention follows—and so does efficient growth.

    If you’re looking for a starting point, try this: define one activation event that clearly signals value, instrument it end-to-end, design a Minimum Path to Value that gets new users there in under 10 minutes, and run weekly experiments until you consistently hit the target. Do that, and you won’t just improve onboarding—you’ll build a product that earns loyalty from the very first session.

    Inspired by this post on Amplitude – Best Practices.

    Book a consult png image
  • Inside Japan’s AI Marketing Shift: How 500 Teams Boost Efficiency, Results, and Careers

    Inside Japan’s AI Marketing Shift: How 500 Teams Boost Efficiency, Results, and Careers

    I just finished reviewing new findings on Japan’s marketing landscape, and the signal is clear: AI isn’t just a shiny tool—it’s a force multiplier for outcomes and careers. The headline that caught my attention, "Amplitude Releases New Research in Japan: Marketers are Unlocking Efficiency, Results, and Career Growth," aligns with what I’m seeing on the ground: teams that blend disciplined analytics with pragmatic AI adoption are pulling ahead.

    Amplitude released a new survey of 500 Japanese marketers, which reveals how teams are benefiting from AI. Get the insights from the data

    Here’s how I interpret the shift. AI accelerates the cycle from insight to action when it’s grounded in a unified analytics platform. With Amplitude analytics stitched into campaign and product signals, marketers can move beyond vanity metrics to diagnose true drivers of activation, engagement, and retention. That’s where efficiency compounds: fewer blind spots, faster iteration, and clearer attribution of what actually drives results.

    On the strategy side, I’m seeing two dominant patterns. First, gen ai is speeding up creative workflows—audience research, message testing, and content generation—without sacrificing brand rigor. Second, agentic AI is emerging in operational loops: routing leads, prioritizing segments, and suggesting next-best actions based on behavioral data. The common denominator is data governance; without clean event schemas and consent-aware pipelines, AI amplifies noise instead of signal.

    For product-led growth motions, this research validates what empowered product teams have practiced for years: instrument the customer journey, frame outcomes vs output OKRs, and experiment in short, learnable cycles. When marketing, product, and data join forces as true product trios, teams can run in-app guides and product tours, tune onboarding, and perform rigorous retention analysis that ties growth to product value rather than spend.

    My playbook in this environment is simple but disciplined. Start with first principles decision making: define the problem, the decision, and the evidence required. Use a unified analytics platform to connect lifecycle events across acquisition, activation, and expansion. Align go-to-market strategy with product roadmapping and sprint planning, so insights move directly into experiments—not slide decks. Then close the loop with clear outcome metrics and QBRs that reward learning velocity, not activity volume.

    There’s also a career arc embedded in this shift. Marketers who cultivate analytical fluency and AI literacy are becoming indispensable partners to product management leadership. They can articulate a differentiated value proposition, shape product positioning with live behavioral data, and influence board-level narratives with credible, causal evidence. That combination—story plus signal—unlocks both performance and professional growth.

    My commitment going forward is to operationalize these lessons: tighter event taxonomy, sharper outcomes framing, and more systematic experimentation across channels and in-product touchpoints. With the right data foundation and a pragmatic AI strategy, we can convert curiosity into capability—and capability into repeatable growth.

    Inspired by this post on Amplitude – Perspectives.

    Book a consult png image
  • Turn Product Insight Into Growth: A Practical Decision Loop

    Turn Product Insight Into Growth: A Practical Decision Loop

    Your funnel says users are leaving during setup. Your survey says they want more features. Sales thinks the positioning is wrong. Each signal may be valid, but none of them is a product decision yet.

    To turn product insight into growth, you need a loop that answers four questions in order: where behavior breaks, why it breaks for a specific group, what small change could alter it, and whether that change improved durable behavior. Skip one, and a plausible idea can consume a sprint without teaching you much.

    Start with the decision your insight must support

    Do not begin with a request to explore the data or understand the customer. Those instructions are too broad. Start with a decision that someone is prepared to make.

    I use a simple test: if the answer cannot change a roadmap choice, an onboarding choice, or an experiment, the question is not specific enough. Write a short decision brief before opening an analytics dashboard or sending a survey.

    • Decision: What choice will this work inform? For example, whether to simplify verification, change setup guidance, or reconsider the activation milestone.
    • Audience: Which users does the decision affect? Separate new users from returning users and identify the relevant channel, plan, role, device, or geography.
    • Outcome: Are you trying to improve activation, feature adoption, or retention? Pick one primary outcome.
    • Unknown: What must you learn before choosing? A location, cause, affected segment, or expected impact is more useful than a general request for feedback.
    • Alternatives: List the realistic actions available. Insight is valuable when it helps you choose among them.
    • Disconfirming evidence: State what would make you reject the leading explanation. This keeps the analysis from becoming a search for support.

    The activation milestone deserves particular care. It should represent the first meaningful value a user receives, not merely an account action that is easy to count. Compare the retention of users who reach a proposed milestone with the retention of those who do not. That cohort contrast can reveal whether the behavior is associated with a more durable relationship. It does not prove causation, but it gives you a stronger milestone to test than intuition alone.

    Do not let feature requests define the decision brief. A request is one expression of a need, filtered through the solution a user happens to imagine. Record it, then identify the underlying job, obstacle, and affected outcome before it reaches the roadmap.

    Use behavioral data to locate the growth constraint

    Behavioral analytics should first tell you where to investigate. It cannot reliably tell you why a user hesitated, but it can narrow a large product journey to a specific transition, cohort, and moment.

    Start with a minimum viable activation map. A useful first pass is four to six events that cover the path from entry to first value. A typical sequence might be sign-up, verification, initial setup, and the first key action. Add an event only when it represents a meaningful state change or helps distinguish between competing explanations.

    Before interpreting the funnel, verify the instrumentation. Use one event taxonomy, consistent names, and properties that let you isolate important groups. Channel, plan, device, role, geography, and cohort are useful when they correspond to a real product or go-to-market decision. An event called setup completed is not trustworthy until the team agrees on exactly what completion means and when it fires.

    1. Build the funnel: Measure completion and drop-off at every transition from entry to first value.
    2. Check event quality: Look for missing properties, duplicate events, unexpected ordering, and definitions that changed between releases.
    3. Segment the loss: Compare channel, device, geography, plan, role, and new versus returning users. A product-wide average can conceal a concentrated problem.
    4. Inspect paths: Look at what users do immediately before and after the weak transition. Repeated steps, detours, and exits help you form a more precise question.
    5. Connect activation to retention: Compare users who reached the milestone with those who did not, then review relevant retention checkpoints.

    Day 1, day 7, and day 30 are useful retention checkpoints alongside lifecycle and unbounded retention views, but they are not universal definitions of success. Match the interpretation to the natural rhythm of your product. A daily workflow and an occasional administrative task should not be judged by the same return pattern.

    Segmentation changes the action. If a drop-off is concentrated on one device, a product-wide tour is likely too broad. If it is concentrated in one acquisition channel, the promise made before sign-up may be attracting users whose expectations do not match the product. If every segment struggles at the same step, the task itself deserves attention before you add more messaging.

    Ask users when the behavioral evidence becomes interesting

    Once the funnel identifies a consequential moment, ask users about that moment. A quarterly survey sent to the entire customer base mixes different jobs, lifecycle stages, and memories. A contextual survey triggered after onboarding, a product tour, or use of a new feature gives the respondent a concrete experience to evaluate.

    Keep the survey small enough to finish. A practical structure is five to seven questions, with two or three quantitative items and one or two open prompts. Use the remaining questions only when they help identify the user’s goal or the obstacle they encountered. Do not ask for profile information already available as product data.

    A five-question diagnostic can look like this:

    1. What were you trying to accomplish?
    2. How confident are you that setup is complete?
    3. How useful was the result you reached?
    4. What, if anything, made the task difficult to complete?
    5. What did you expect to happen next?

    The first question identifies the job. The two rating questions create trendable measures. The open prompts expose vocabulary, expectations, and failure modes that predefined answer choices can miss. Adjust the wording to the actual moment; do not ask someone who abandoned setup to rate a result they never saw.

    Target cohorts separately. New users can explain expectation and comprehension gaps. Power users can expose workflow limitations. Retained and churning users can describe different value patterns. Combining them into one score produces an average that may represent none of them well.

    Tell people why you are asking, how long the survey will take, and how the response will inform a decision. Then close the loop by sharing what changed. This is not ceremonial communication. It gives users evidence that thoughtful feedback does not disappear into a backlog.

    For a large volume of open text, generative AI can accelerate initial clustering and sentiment labeling. Treat that output as a sorting aid, not a conclusion. Validate the themes manually and compare them with product telemetry. Models can merge comments that use similar language but describe different jobs, or separate comments that describe the same obstacle in different words.

    Survey respondents are also a selected group: they were available and willing to answer. Compare their behavior with the full target cohort before generalizing. If respondents complete setup far more often than nonrespondents, their explanation may not represent the users you most need to understand.

    Triangulate evidence instead of letting signals vote

    Behavior and feedback do not need to agree perfectly. Their job is to constrain the explanation. Telemetry shows what happened at scale. Contextual feedback supplies possible reasons. Retention indicates whether the behavior mattered beyond the immediate session.

    Behavioral signalUser feedbackInterpretation to testNext move
    Users stall before the key actionThey report an unclear next stepComprehension or discoverability may be blocking progressTest clearer guidance at the exact transition
    Users stall before the key actionThey describe an error or failed dependencyExecution friction may matter more than educationFix the failure before adding tours or tooltips
    Users complete the funnelThey rate the outcome as having low usefulnessThe milestone may measure activity rather than valueRevisit the activation definition and value proposition
    Users reach first value and rate it highlyLater retention remains weakThe problem may occur after activationAnalyze the post-activation path and repeat-value moments

    Use each row as a hypothesis, not a diagnosis. The same behavioral pattern can have several causes. A user might leave verification because the instructions are unclear, because the task fails, because the requested information feels unnecessary, or because the value promised before sign-up was not compelling enough. The next evidence or experiment should distinguish among those explanations.

    Translate the combined evidence into a problem statement before discussing solutions:

    When [specific cohort] tries to [job], they stall at [event or transition]. We observe [behavioral evidence], and contextual feedback repeatedly describes [theme]. This appears to affect [activation, adoption, or retention outcome].

    This format prevents a popular feature request from outranking a larger but less vocal obstacle. Rank the resulting opportunities by user impact, strategic fit, and strength of evidence. Then connect each selected opportunity to a measurable activation, adoption, or retention outcome rather than treating delivery as success.

    Conflicting evidence is useful when you investigate the conflict. High reported ease alongside high funnel abandonment may indicate respondent bias, a faulty event definition, or a hidden segment with a different experience. High activation among completers alongside severe pre-activation loss may point to an onboarding gate around a valuable product. Those patterns lead to different decisions, even if the top-line conversion rate is identical.

    Convert one insight into a testable growth bet

    An insight is not finished when it becomes a presentation. It is finished when it changes a decision and creates a measurable test. Capture the bet in one experiment card:

    • Problem: The cohort, job, and transition described in the problem statement.
    • Hypothesis: The mechanism you believe is causing the observed behavior.
    • Change: The smallest intervention that tests that mechanism.
    • Audience: The exact users who should encounter the change.
    • Primary metric: The activation, adoption, or retention behavior expected to move.
    • Guardrail: A behavior that should not deteriorate while the primary metric improves.
    • Evaluation: How you will distinguish the effect of the change from ordinary variation.
    • Next decision: What you will do if the result is positive, neutral, or negative.

    Match the intervention to the suspected mechanism. An in-app guide can help a user resume a setup sequence. A product tour can expose a core workflow that users consistently overlook. A tooltip can resolve uncertainty at one control or decision point. None of them will repair a broken task, a misleading acquisition promise, or a weak value proposition.

    Prefer a focused change over a wholesale onboarding redesign because it gives you a clearer learning signal. When traffic and risk allow, compare the changed experience with an appropriate control. Define the success measure before launch. Do not declare victory from higher setup completion if users still fail to reach first value or if the relevant retention behavior does not improve.

    Put the bet into normal product roadmapping and sprint planning, and keep the evidence visible on a shared dashboard. Product, engineering, design, customer support, and customer-facing technical roles each see a different part of the journey. Their observations should refine the hypothesis, while the agreed metric remains the arbiter of the result.

    When the decision is made, update the insight record with the result: observed, validated, tested, adopted, or rejected. Share the outcome with the users who contributed feedback when practical. Closing both the analytical loop and the communication loop makes the next round of discovery easier.

    Key takeaways

    • Define the decision, cohort, outcome, and disconfirming evidence before collecting more data.
    • Map four to six trustworthy events from entry to first value, then segment the weak transition.
    • Use retention to check whether the proposed activation behavior is associated with durable value.
    • Trigger a five-to-seven-question survey at a meaningful product moment and combine ratings with open prompts.
    • Treat telemetry and feedback as inputs to a hypothesis, not competing votes on the roadmap.
    • Ship the smallest intervention that tests the suspected mechanism, then measure the downstream behavior that matters.

    If you have one hour, choose one activation journey, verify the four to six events that describe it, segment new and returning users, and identify one consequential drop-off. Write one problem statement and one experiment card before refining the dashboard. That is enough to turn a vague growth discussion into a decision the team can act on.

    References