Customer feedback is the most reliable compass I have for product strategy and execution. Over the years leading product at HighLevel, I’ve built and refined a system that turns raw signals from users into clear, prioritized decisions our teams can confidently ship.
A practical guide to collecting and using product feedback in product management (from AI tools to early-stage tactics) for better product decisions.
My playbook starts with continuous discovery. I keep a steady flow of insights from sales calls, customer support threads, community forums, and in-product behavior so I can triangulate patterns rather than chase loud anecdotes. This mix of quantitative and qualitative data helps me separate urgent noise from strategically meaningful trends.
On the quantitative side, I rely on product analytics to ground the conversation. Amplitude analytics gives me activation, retention cohorts, and feature engagement, while controlled experiments and A/B testing validate whether an idea actually moves a target metric. Tying these signals to specific customer segments helps me see where product-led growth is working—and where it’s stalling.
For qualitative insight, I combine in-app guides and lightweight surveys (via tools like Pendo) with structured interviews and support escalations (often surfaced through platforms like Intercom). I map problems using the Kano Model to understand which requests are basic expectations, which are performance drivers, and which are potential delights. This keeps our roadmap focused on outcomes, not just outputs.
AI now accelerates the synthesis step. With LLMs for product managers in my AI product toolbox, I summarize interview transcripts, cluster themes across thousands of notes, and quantify sentiment without losing nuance. I still review raw artifacts to avoid hallucinations and preserve context, but AI reduces the time from signal to insight dramatically—freeing me to spend more energy on judgment and storytelling.
In early-stage contexts, I bias toward speed and proximity to users. I schedule founder- or PM-led discovery calls weekly, instrument product tours early, and launch scrappy in-product prompts to validate demand before over-investing. When data is sparse, I focus on high-signal channels (power users, churned customers with qualified use cases) and document crisp problem statements that connect directly to activation, retention analysis, and revenue outcomes.
Prioritization ties everything together. I translate insights into hypotheses aligned to outcomes vs output OKRs, then pressure-test them with feasibility and strategic fit. We run small, measurable experiments, track deltas in activation and retention, and adjust the product roadmapping and sprint planning cadence based on what the data and customers teach us.
This approach builds trust with stakeholders and creates empowered product teams. By grounding decisions in a transparent trail of feedback, analytics, and experiments, we reduce thrash, move faster, and—most importantly—ship product moments that customers value.
If you’re refining your own feedback engine, start by instrumenting the basics, set a weekly discovery rhythm, and let AI handle the heavy lifting on aggregation and synthesis. The compounding effect is real: better insights lead to better bets, which lead to better outcomes for your users and your business.
Your contact center can have excellent reporting and still react too late. A weekly chart may explain why transfers rose, authentication failed, or members called again. It cannot recover the interaction that is already going wrong.
That is the practical case for real-time analytics in financial services: detect a useful signal while there is still time to change the outcome, then deliver a safe action to the person or system that can take it. The goal is not a faster dashboard. It is a shorter path from behavior to decision to resolution.
Key takeaways
Define real time against the decision window. A signal is timely only if it arrives before the next useful action expires.
Start with journeys that create material cost or dissatisfaction, such as lost cards, fraud disputes, loan-status requests, password resets, and payment issues.
Instrument the outcome as carefully as the interaction. Otherwise, you can see that an alert fired without knowing whether it helped.
Activate insights inside routing, agent, supervisor, and follow-up workflows. A separate analytics destination creates another queue for people to monitor.
Measure resolution, repeat demand, and guardrails. Activity metrics such as alerts generated or prompts displayed are diagnostics, not business outcomes.
Build privacy controls, consent handling, access restrictions, and auditability into the decision loop before expanding its reach.
Define real time as a decision contract
Real time is not a universal refresh rate. It is a promise that a signal will reach its decision point while an effective response is still possible. An agent-assist prompt must arrive before the conversation moves past the relevant step. A routing signal must arrive before the interaction enters the wrong queue. A proactive follow-up must arrive before the member has to contact you again.
This distinction prevents an expensive architecture mistake: streaming every event without deciding what any event should change. Some information needs immediate activation. Some belongs in a supervisor review. Some is useful only for longer-term journey redesign. Treating all three as equally urgent increases cost and noise without improving service.
Before building a pipeline, write a decision contract for each use case. The contract should connect the signal to an owner, action, deadline, guardrail, and measurable outcome.
Decision-contract field
Question to answer
Illustrative fraud-routing example
Trigger
What observable event or state starts the decision?
A potential fraud signal appears during an active interaction.
Decision
What choice becomes possible because of the signal?
Whether the interaction should receive specialized handling.
Action
What should the workflow do?
Prioritize the appropriate route and carry the available context forward.
Owner
Who or what is accountable for acting?
The routing workflow, with a supervisor responsible for defined exceptions.
Action window
When does the intervention stop being useful?
Before the interaction is transferred or the relevant verification step is completed.
Guardrail
What must never be bypassed?
Required compliance steps, authorized data access, and a clear human override.
Outcome
How will you know whether the action helped?
Resolution without an avoidable transfer, escalation, or repeat contact.
A contract also exposes weak use cases early. If nobody can name the action, the signal is probably reporting data rather than real-time decision data. If the action has no owner, it will become an ignored alert. If the outcome is merely that a prompt appeared, the team has confused delivery with impact.
The underlying platform still needs to bring together behavior across voice, chat, IVR, email, and in-app journeys. But unification is useful only when identity, journey state, and timing remain coherent across those channels. A member who fails authentication in the app and then calls should not look like two unrelated problems.
Instrument five costly journeys before the whole contact center
A complete contact-center data program is too broad a starting point. It invites months of taxonomy work before anyone changes an outcome. Begin with the five journeys most likely to concentrate cost or dissatisfaction: lost card, fraud dispute, loan status, password reset, and payment issue.
This is not a mandate to automate all five at once. Rank them using the evidence you already have: contact demand, transfers, repeat contacts, unresolved cases, authentication failures, and escalations. Choose the journey where a specific intervention is both valuable and operationally feasible.
For the chosen journey, create an outcome card before defining events:
Member intent: What is the person actually trying to complete?
Observable start: Which event shows that the journey has begun?
Resolution state: What evidence means the need was completed, not merely that the interaction ended?
Failure states: Where can authentication, routing, handoff, self-service, or follow-up break down?
Intervention: Which failure can the contact center change while the journey is active?
Outcome and guardrails: Which result should move, and which compliance or experience measures must not deteriorate?
The event model should then describe the journey rather than mirror the screens of each tool. At minimum, preserve a pseudonymous member reference, interaction reference, channel, event time, journey, journey step, authentication state, transfer or escalation state, intervention, and outcome. If intent or risk is inferred, record the version and confidence associated with that inference. If an agent accepts, dismisses, or overrides guidance, capture that response too.
Consistent definitions matter more than a large event count. Decide what a transfer is, when a new contact belongs to an existing journey, and what qualifies as resolution. Version those definitions. Otherwise, a changed IVR flow or CRM configuration can appear to improve performance simply because the instrumentation changed.
Instrument the negative space as well. If the member disappears from a self-service flow, the absence of a completion event is not enough to explain why. Capture the last meaningful step, the failure category when it is available, and whether the member moved to another channel. That is how you distinguish successful deflection from abandonment followed by a call.
Do not copy every transcript, recording, credential, or financial value into a broadly accessible analytics stream merely because the technology allows it. Use minimized attributes and controlled references where they are sufficient. Keep restricted evidence behind narrower permissions. Availability is not the same as permission.
Put the decision inside the workflow
The last mile determines whether real-time analytics changes performance. An insight that requires an agent to open another application, interpret a graph, and decide what it means has already lost much of its value. Activation belongs in the systems where agents, supervisors, and automated workflows already act.
Four activation patterns cover most of the useful surface area:
Agent guidance: Surface the next relevant step, missing compliance action, or known journey context during the interaction. Explain why the guidance appeared, avoid conflicting prompts, and give the agent a defined way to dismiss or override it.
Supervisor intervention: Alert on a material pattern with an attached playbook. The notification should identify what changed, which interactions are affected, which action is available, and when the alert expires.
Member follow-up: Trigger a relevant message or next step after an unresolved interaction. The follow-up should close a known gap, not merely create another generic communication.
Self-service requires particular care. If balance inquiries or password resets are overwhelming queues, routing eligible demand to self-service may help. But containment is not the same as resolution. Measure whether the member completed the task and whether another contact followed. A journey that exits the IVR but returns through chat has changed channels, not disappeared.
Each activation needs a safe fallback. If identity is uncertain, the signal is stale, or a dependency is unavailable, revert to the normal approved workflow. Do not let a broken analytics path invent a route or compliance step. Log the fallback so operational teams can distinguish a bad recommendation from a recommendation that never reached its destination.
Alert design deserves the same product discipline as customer-facing design. Deduplicate repeated signals, suppress guidance after the relevant action window, and route exceptions to a named owner. A queue full of low-value alerts trains people to ignore the important ones.
The technology choice comes after these workflow requirements. CRM integration should carry member and journey context forward, while the analytics layer captures behavior and evaluates interventions. Products such as Amplitude, Pendo, and Intercom may instrument digital touchpoints, but the build-versus-buy decision should turn on your decision contracts: identity reconciliation, activation latency, workflow integrations, experimentation, access control, auditability, and operational reliability.
I would not approve a platform solely because its dashboards are polished. Ask the vendor or internal platform team to demonstrate an end-to-end loop using one of your journeys: signal received, decision evaluated, workflow changed, outcome captured, and audit record produced. That sequence is the product you are buying or building.
Measure outcomes, experiment carefully, and govern the loop
Real-time analytics does not reduce operating cost by itself. It changes a decision, which changes a journey, which may change demand and resolution. Your measurement model has to preserve that chain.
Use a scorecard that separates outcomes from activity
Journey outcome: Was the member’s need resolved, and did it stay resolved?
Operational mechanism: Did transfers, escalations, routing failures, or authentication failures change?
Intervention delivery: Was the recommendation generated, delivered in time, accepted, dismissed, or overridden?
Experience and compliance guardrails: Were required steps completed, and did complaints, corrections, or manual exceptions increase?
System health: Was the signal complete, timely, correctly joined to the journey, and available when the workflow needed it?
Average handle time can be diagnostic, but it should not become the automatic objective. A shorter interaction that leaves the member unresolved may simply move cost into a repeat contact. Resolution and repeat demand tell you whether the system removed work or postponed it.
Test the intervention, not the existence of the data
Controlled experiments can show whether a changed IVR path, authentication step, or post-contact follow-up improves the chosen outcome. Define the minimum detectable effect before the test so the team knows which improvement would justify a decision and whether the eligible volume can support a useful result.
Choose the unit of assignment deliberately. If the same member can return during the measurement window, assigning different experiences by interaction can contaminate the comparison. A member-level assignment may be cleaner. If the intervention changes an entire queue or supervisor workflow, individual assignment may be impractical; use a rollout design that reflects how the operation actually works.
Do not randomize away mandatory compliance controls. When an intervention affects fraud handling, sensitive disclosures, or consequential routing, begin in observe-only mode, review false positives and overrides, and use an approved rollout. Experiment with the delivery or operational design only where compliance and legal owners confirm that variation is permissible.
Make governance part of the product
Privacy and compliance cannot sit downstream of activation. A real-time system makes decisions from live member behavior, so access controls, consent management, and audit trails belong in the initial architecture.
For every decision contract, document the permitted purpose of the data, who can access it, where it is retained, how consent is honored, what enters the audit record, and who approves changes. Do not infer that an attribute is lawful to use because it exists in the CRM. The relevant compliance and legal owners must determine acceptable use for the jurisdiction, product, and member context.
Auditability should reach beyond data access. Preserve enough context to reconstruct what signal arrived, which rule or model version evaluated it, what action was recommended, what the workflow did, whether a person overrode it, and what outcome followed. That record supports incident investigation, performance review, and defensible change management.
Run the operating cadence through a product trio spanning operations, data, and compliance. In each review, ask which decisions fired, which arrived too late, which actions were ignored, which outcomes changed, and which guardrails moved. Retire noisy signals. Refine ambiguous definitions. Promote successful interventions gradually. This keeps the program focused on decision quality instead of dashboard volume.
Your next step is small and concrete: choose the highest-cost or highest-friction journey among the initial five, write its decision contract, and run the signal in observe-only mode. When the team can trace the path from trigger to approved action to outcome, activate the narrowest useful intervention. Expand only after that loop is measurable, reliable, and governable.
Your roadmap can be coherent, the launch can go smoothly, and adoption can still stall. The usual gap is not a lack of effort. Product strategy, field discovery, technical validation, and post-launch adoption were treated as separate activities, so each team learned something that the others could not use.
You can close that gap by treating solutions engineering as part of the product learning system. The goal is not to give sales more technical coverage or product managers a larger queue of requests. It is to turn real customer conditions into testable product decisions, then follow those decisions until customers reach and repeat the intended outcome.
Key takeaways
Define adoption as an observable customer behavior before you approve features or plan a launch.
Ask solutions engineers to capture evidence in a consistent format, separating what happened from what the team thinks it means.
Treat demos and proofs of value as experiments with a hypothesis, baseline, value event, and decision rule.
Measure the entire path from promise to repeated value. A login, click, or feature visit is rarely sufficient evidence of adoption.
Route each adoption problem to the right response: positioning, enablement, onboarding, integration, product design, or core capability.
Carry launch learning into roadmap and operating reviews so that adoption changes product decisions instead of becoming a reporting exercise.
Write the adoption contract before debating the roadmap
Product strategy becomes operational only when it specifies whose behavior should change, why that change matters, and what evidence would show that the product created value. Without that translation, teams can agree on a strategic theme while holding incompatible ideas about what success means.
Write an adoption contract for every significant product bet. This is not a legal document or a long requirements package. It is a compact agreement among product, engineering, design, solutions engineering, sales, and customer success about the outcome being pursued.
Target user: Name the persona and operating context. A broad account segment is not enough when different people inside the account perform different jobs.
Starting condition: Describe the workflow, constraint, or workaround that exists before the change. This gives you a baseline against which the new experience can be judged.
Desired outcome: State the progress the customer wants, not the capability you intend to ship.
Value event: Identify the observable behavior showing that the user reached meaningful value. Opening a page may indicate exposure; completing the intended workflow is stronger evidence.
Repeat condition: Define what sustained use looks like at the natural cadence of the workflow. Daily activity is useful only when the job itself occurs daily.
Decision evidence: Agree on the metric, qualitative evidence, technical constraints, and guardrails that will determine whether to continue, change, or stop the bet.
Suppose a product automates part of an operations workflow. Increase feature adoption is too weak to guide a team. A more useful contract says that the operations manager can complete the target workflow, obtain the intended result, and return to it when the job recurs without relying on the old workaround. The team can now examine setup, permissions, integrations, comprehension, completion, and repetition as separate parts of the same outcome.
This contract also changes roadmap conversations. A feature request no longer competes on enthusiasm alone. You can ask which persona it serves, which blocked outcome it unlocks, what evidence supports the problem, and which customer behavior should change if the solution works. If those questions have no answers, the request needs more discovery before it needs an estimate.
Keep the contract stable enough to align the organization, but do not protect it from evidence. If customers repeatedly pursue a different outcome, struggle with an unanticipated dependency, or assign value to another part of the workflow, revise the contract explicitly. Silent reinterpretation is dangerous because every function can declare success against a different definition.
Turn solutions engineering into a field evidence system
A solutions engineer sees details that rarely survive a conventional feature request: the customer’s existing architecture, the sequence of the workflow, the people involved in approval, the point at which a demonstration loses credibility, and the difference between a stated objection and an actual blocker. That information can improve strategy, but only if the organization captures it in a form that product teams can compare and act on.
Do not send raw call notes into a backlog and call it discovery. Use a consistent field record containing:
The persona, account context, and job being attempted.
The customer’s desired outcome and current way of achieving it.
The specific moment where the current product or proposed solution created friction.
The relevant technical condition, such as an integration, data, permission, security, or scalability constraint.
The evidence observed: customer behavior, workflow review, demonstration response, deployment result, or instrumented product data.
The proposed interpretation and any plausible alternative explanation.
The next question or test that would reduce uncertainty.
Within that record, separate observation, interpretation, and request. They are not interchangeable.
Observation: The administrator could not complete configuration because a required permission was unavailable.
Interpretation: The permission model may not match how this persona operates.
Request: Add a new administrative role.
The observation is evidence. The interpretation is a hypothesis. The request is merely a candidate solution. Keeping them separate prevents the first proposed feature from becoming the definition of the problem.
Product leaders also need to protect this system from deal gravity. A commercially urgent request can be important without representing a widespread product problem. Conversely, a small implementation detail can reveal a structural barrier affecting an entire target persona. Evaluate field evidence with questions that make those differences visible:
Does the problem recur for the same persona, workflow, or environment?
Does it block the value event, or does it affect a peripheral preference?
Is the root cause a missing capability, product usability, configuration, integration, positioning, or customer process?
Does the problem appear in the strategic segment the product is designed to serve?
Can a smaller or reversible test distinguish competing explanations?
What roadmap decision would change if the hypothesis were confirmed?
The last question is a useful filter. If no plausible answer would change a decision, the team may be collecting interesting information rather than decision-grade evidence.
Bring the strongest records into the product trio’s learning cycle. Field evidence can then sharpen roadmaps, sprint planning, positioning, integration decisions, and stakeholder conversations. The solutions engineer contributes technical discovery and customer context. Product management synthesizes patterns and owns tradeoffs. Design examines behavior and workflow. Engineering tests feasibility and exposes architectural consequences. Sales contributes commercial context without converting urgency into automatic priority. Customer success adds evidence about repeated use after the initial deployment.
This arrangement does not require every customer conversation to become a committee meeting. It requires a common evidence format, clear decision ownership, and a route by which meaningful field learning reaches the people making product choices.
Run demos and proofs of value as product experiments
A polished demo can prove that a presenter understands the product. It does not prove that a customer can adopt it. Even a successful technical pilot can produce a false positive if the solutions engineer performed work that the target user cannot repeat, the environment was unusually controlled, or the team never defined what customer value would look like.
Design every proof around a falsifiable outcome hypothesis. A practical structure is:
For [persona] in [context], enabling [change] should produce [observable behavior] because [value mechanism]. The hypothesis is supported if [agreed evidence] meets [predefined decision rule] without violating [guardrail].
The brackets force useful decisions. You have to name the user, the operating condition, the expected behavior, and the reason the behavior represents value. You also have to decide what would count as failure before enthusiasm about the result changes the standard.
Build the proof plan around the following elements:
Baseline: Document how the customer handles the job now, including meaningful friction and dependencies.
Scope: State which workflow, persona, environment, and constraints the proof includes. Anything outside that boundary remains unvalidated.
Value event: Identify the customer action or result that demonstrates progress toward the desired outcome.
Instrumentation: Decide which product events and qualitative observations will show where users advance or stop.
Enablement: Record how much assistance the customer receives. Heavy expert intervention can validate technical feasibility while leaving self-service adoption unproven.
Decision rule: Define what evidence will justify progression, iteration, repositioning, or stopping.
Learning owner: Name the person responsible for translating the result into a product, go-to-market, or adoption decision.
A compact learning record can follow Insight – Hypothesis – Experiment – Metric. The sequence matters. A customer comment produces an insight, not a roadmap commitment. The insight leads to an explanation that can be tested. The test produces evidence against a metric or decision criterion.
Capture objections with the same discipline. An objection about missing functionality may actually reflect an unclear value proposition, concern about integration effort, lack of trust in the output, or a mismatch between the buyer and the eventual user. Ask what the customer is unable or unwilling to do because of the concern. That question moves the discussion from the requested feature to the blocked behavior.
At the end of a proof, classify what was learned:
Value validated: The target persona reached the intended outcome under representative conditions.
Need validated, capability missing: The problem matters, but the product cannot yet support the required workflow.
Capability works, adoption friction remains: The product can deliver value, but setup, comprehension, trust, or workflow design prevents the user from reaching it reliably.
Technical path blocked: An integration, architecture, data, permission, or operational constraint prevents deployment.
Positioning mismatch: The product can do what was promised, but the outcome is not important enough for the target persona or was framed in the wrong terms.
Evidence inconclusive: The test did not represent the intended user or environment, or the decision criteria were not measurable.
These outcomes lead to different work. A positioning mismatch belongs in messaging and discovery. Adoption friction may require onboarding or product design. A recurring technical blocker may deserve roadmap investment. An inconclusive test deserves a better test, not a success story.
Demos and early deployments become high-signal learning mechanisms when their evidence returns to product strategy instead of ending in the opportunity record. That closed loop is what allows solutions engineering to improve both customer execution and the underlying product.
Measure adoption as a path, then act at the break
Adoption is not a single metric. It is a sequence that begins with a credible promise and ends when the customer repeatedly receives value. Measuring only the end hides the reason users failed. Measuring only clicks overstates progress.
Map the path for the persona named in the adoption contract:
Promise understood: The user or buyer can connect the product to a relevant outcome.
Entry: The intended user begins setup or enters the target workflow.
Readiness: Required data, permissions, integrations, and configuration are in place.
Activation: The user reaches the first meaningful value event.
Repetition: The user returns when the underlying job recurs and reaches value again.
Expansion: Appropriate users, workflows, or capabilities are added because the initial value is credible.
Retention: The product remains part of the operating workflow because it continues to produce the desired outcome.
Instrument enough of this path to locate the break. For each event, retain the eligible persona or account, relevant product context, action attempted, result, and connection to the intended outcome. Always state the denominator. Activation among configured users answers a different question from activation among all purchased accounts.
Use metrics such as time-to-value, daily active usage, and feature adoption only when they match the product’s natural workflow. A monthly administrative task should not be judged by daily activity. A high feature-visit count does not establish value if users repeatedly enter the feature because they are confused. Pair behavioral data with the qualitative context collected during proofs, implementations, and customer conversations.
Observed pattern
Question to investigate
Likely response area
Interest is high, but eligible users do not begin
Is the value proposition relevant and credible to this persona?
Positioning, targeting, or enablement
Users begin, but readiness fails
Which data, permission, configuration, or integration dependency is blocking progress?
Onboarding, implementation, or platform capability
Users complete setup, but do not reach the value event
Does the workflow lead clearly to the promised outcome?
Product design, guidance, or core capability
Users activate, but do not return
Is the job recurring, and was the first outcome valuable enough to change behavior?
Discovery, workflow fit, trust, or value proposition
Adoption differs sharply by persona or context
Was the product designed and positioned for the segment that is actually succeeding?
Segmentation, strategy, or packaging
Usage is high, but retention or customer outcomes remain weak
Is activity measuring value, required effort, or repeated friction?
Metric design, product quality, or outcome alignment
Match the intervention to the break. Use onboarding checklists, empty-state prompts, in-app guides, and product tours when the user needs contextual direction. Do not use a tooltip to conceal a broken permission model, an unreliable integration, or a workflow that does not create enough value. Guidance can reduce comprehension friction; it cannot repair a missing capability.
When you test an intervention, define the intended behavior change and success criterion in advance. Segment the result by the persona and context in the adoption contract. An aggregate improvement can hide a deteriorating experience for the strategic user, while a neutral aggregate can hide a strong response in the segment the product is meant to serve.
The adoption contract and any evidence that challenges it.
Progress through the journey, segmented by the intended persona and context.
The strongest field observations, with interpretations kept separate.
The active hypotheses and experiments.
The product, positioning, enablement, integration, or onboarding decision each experiment may change.
The owner responsible for making that decision and returning the result to the group.
Use operating reviews and QBRs to evaluate outcomes, not to count shipped features or completed guides. Ask which hypothesis was confirmed, which constraint was removed, where customers still stop, and what the organization will do differently. If an experiment has no route to a decision, revise it or stop it.
For your next significant product bet, start before the launch plan. Write the adoption contract, ask a solutions engineer to challenge it with field conditions, and define the evidence that would change your mind. That small discipline gives every later conversation – roadmap, demo, onboarding, analytics, and QBR – the same customer outcome to pursue.
Every quarter, I watch product teams move from gut feel to data-informed decisions—until instrumentation bottlenecks slow them to a crawl. That’s why I’ve become an advocate for codeless analytics: it removes the dependency on engineering sprints for basic event tracking and lets teams answer product questions in hours, not weeks.
We explain what codeless analytics are, why (and how) Pendo supports them, plus responses to the top three myths about low-code/no-code solutions.
Here’s how I frame it with my teams: codeless analytics enables product managers, designers, and customer success to tag features visually, track interactions, and analyze adoption without shipping code. The goal isn’t to replace engineered events; it’s to accelerate discovery, speed up iteration, and reduce context-switching for developers. In practice, this means cleaner prioritization, faster validation of hypotheses, and tighter product-led growth loops.
Why Pendo? In my experience, Pendo’s codeless model shortens the distance from question to insight. Visual tagging makes event setup accessible, in-app guides and product tours let us experiment with onboarding and activation, and governance controls ensure data remains trustworthy across teams. The result is a unified analytics approach where we reserve custom instrumentation for complex logic while using codeless tracking for everyday product questions.
Let’s address the top three myths I hear most often. Myth 1: “No-code is only for simple use cases.” In reality, most decisions we make weekly—feature adoption, path analysis, funnel drop-offs, and retention analysis—do not require custom code. Codeless analytics handles these well, and when we need deeper context (like server-side events), we complement it with engineered tracking. It’s a both/and, not an either/or.
Myth 2: “Codeless data isn’t accurate.” Accuracy comes from governance, not the method. I set clear standards: naming conventions, tagging reviews, ownership, and periodic audits. With disciplined process, codeless tracking yields consistent, decision-grade data. The added benefit is visibility—non-technical stakeholders can validate the instrumentation themselves, reducing misalignment.
Myth 3: “Engineers must instrument everything to scale.” Engineering time is precious; we should spend it on differentiated capabilities, not on routine click tracking. Codeless analytics scales by empowering product teams to self-serve, while engineering focuses on back-end, performance, and edge cases. When paired with a unified analytics platform and clear data contracts, this model scales cleanly across product lines.
For teams adopting this approach, I recommend a simple operating model: define your core product questions up front, tag features aligned to those questions, connect insights to in-app guides for experiments, and measure user activation and retention continuously. Whether you run Pendo alongside Amplitude analytics or within a broader unified analytics platform, the key is to keep the insight-to-action loop tight.
The future of product analytics is codeless because it puts insights where they belong—directly in the hands of the people designing the experience. When we remove bottlenecks, we learn faster, ship smarter, and drive measurable PLG impact. That’s how we turn product analytics from a reporting function into a competitive advantage.
Trust is the currency of any high-stakes AI product, and nowhere is that more true than in healthcare. I recently dug into how Healio built an AI assistant for physicians—an audience that can’t afford to be wrong—and it’s a masterclass in balancing accuracy, transparency, and speed without compromising credibility.
Healio, a 125-year-old medical publishing company, set out to create Healio AI to help clinicians prepare for patient care. From the outset, their guiding principle was simple: physicians won’t trust you until you prove it. That lens shaped every decision—from discovery and prototyping to architecture, evaluation, and ongoing validation.
Discovery started with a survey of 300 healthcare professionals to understand real-world needs at the point of care. The headline insight: physicians primarily want AI for preparation, not bedside use. Even more surprising, the top ask wasn’t purely diagnostic support; it was help with patient communication and empathy—translating complex information into clear, accessible conversation.
Momentum mattered. After beginning with Figma mockups to validate workflows, the team built a working prototype in a single weekend using Cursor. That velocity wasn’t about cutting corners; it was about proving value quickly, reducing ambiguity, and iterating with concrete feedback from physicians.
Under the hood, the system employs RAG and hybrid search—combining lexical search, vector search, and semantic search across multiple trusted sources like PubMed. As any PM who has integrated biomedical literature knows, "just use PubMed" isn’t simple—there are five different ways to access the same data, each with trade-offs. The team made pragmatic choices to balance freshness, coverage, latency, and cost while preserving trust in source quality.
Designing for trust extended all the way to the citation UX. The team leaned into citations that physicians actually trust: subscripts, hover states, and progressive disclosure. This gave clinicians verifiable threads back to source material without overwhelming the core interaction, aligning with how experts want to audit evidence under time pressure.
Evaluation wasn’t left to chance. They stood up eight LLM judges for evals: safety, medical accuracy, faithfulness, relevancy, completeness, reasoning, clarity, and overall quality. Just as importantly, they treated those signals as directional, not definitive. In a high-stakes domain, physician feedback trumps LLM-as-judge feedback—so they complemented automated evals with direct reviews from practicing clinicians to calibrate quality and reduce hallucinations.
On the safety front, the team implemented HIPAA compliance and input guardrails for masking personal health information. That choice reflects strong data governance and privacy-by-design thinking: protect PHI by default, constrain prompts to safe boundaries, and make compliance a first-class citizen in the product architecture.
They also addressed monetization without compromising experience. Serving contextual ads while the LLM processes queries is a practical approach that preserves physician workflow efficiency and creates a clear, non-intrusive revenue model.
Critically, the work didn’t stop at launch. The Healio Innovation Partners provide ongoing discovery and validation, ensuring the system evolves with physician needs and the medical evidence base. This is the operating cadence you want for any AI product that sits at the intersection of safety, accuracy, and fast-changing knowledge.
My takeaways for building AI in high-stakes domains: prioritize retrieval-first pipelines over model cleverness; couple RAG with hybrid search across vetted sources; design citations that earn trust at a glance; use eval-driven development, but let domain-expert feedback be the ultimate judge; and embed regulatory compliance into your product strategy from day one. If trust is your North Star, this is a playbook worth emulating.
Across my teams and portfolio, I’m watching AI fundamentally reshape product-led growth—from static funnels and one-off playbooks to adaptive, compounding growth loops that learn in real time. The shift isn’t just technological; it’s an operating model change that rewards continuous discovery, rigorous instrumentation, and outcome-driven product strategy.
"Learn how AI is transforming PLG with a new generation of growth loops that can turn your product into a self-optimizing platform." That line captures what I’ve been building toward: systems that sense user intent, decide the next best action, act contextually, and learn to improve the loop with every interaction.
Here’s the core pattern I rely on. First, sense: unify product analytics and behavioral signals (think Amplitude analytics, Pendo events, Intercom conversations) into a single, queryable, privacy-safe layer. Second, decide: apply AI Strategy—LLMs for product managers, rules, and retrieval—to segment users by intent and probability of success. Third, act: deliver in-app guides, product tours, tooltips, or personalized nudges that accelerate user activation and time-to-value. Finally, learn: run A/B testing with a clear minimum detectable effect (MDE), then feed outcomes back into the model for continuous optimization.
Activation is where the gains start compounding. With gen ai, I can auto-generate tailored onboarding checklists, dynamic walkthroughs, and contextual help that adapts to the user’s role, data maturity, and current friction points. We’ve moved from generic product tours to precision guidance that updates based on real-time behavior—often lifting first-week activation and shortening time-to-first-value without adding support load.
Experimentation is the governor that keeps speed and quality in balance. I instrument every growth loop end to end and pair eval-driven development with A/B testing to confirm incremental impact. Amplitude analytics gives me cohort views and path analysis; Pendo or Intercom can deliver in-app variants; a unified analytics platform closes the loop on retention analysis so I’m not optimizing for click-through at the expense of long-term value.
Retention and expansion are where AI shines as a compounding engine. Retrieval-first pipeline patterns allow instant, contextual support that deflects tickets and boosts perceived product competence. Agentic AI can orchestrate next-best actions—prompting power users toward advanced features, surfacing value moments, or timing expansion prompts when success signals appear. The result is a virtuous cycle: better guidance drives deeper adoption, which improves model accuracy, which unlocks more relevant guidance.
None of this works without guardrails. I bake in AI risk management from the start: strict data governance, privacy-by-design, human-in-the-loop review for high-impact actions, transparent user consent, and continuous drift monitoring. The goal is reliable automation that users trust—augmented by clear fail-safes when confidence drops.
Operationally, I anchor the work in empowered product teams and product trios, focus on outcomes vs output OKRs, and practice continuous discovery to validate problems and solutions before scaling. The baseline metrics I watch: activation rate, time-to-value, week-four retention, PQL/PQA conversion, expansion revenue, and support deflection—each tied to a specific growth loop hypothesis.
If you’re starting fresh, begin with the highest-leverage loop: user activation. Instrument your onboarding journey, define the critical path to value, ship two to three personalized interventions, and measure impact with a precommitted MDE. Scale what wins, drop what doesn’t, and iterate weekly. Once activation is compounding, extend the same approach to adoption depth, collaboration features, and expansion triggers.
In practical terms, AI-powered PLG is less about flashy features and more about disciplined feedback loops. Build the sensing fabric, keep the decision layer auditable, ship small actions quickly, and treat learning as the product. Do that, and your product doesn’t just grow—it becomes a self-optimizing platform.
I just tuned into the latest conversation on the upcoming Product at Heart 2026, and it hit on the exact challenges product leaders are navigating right now: curating meaningful content in a world where AI moves faster than our agendas, designing formats that create real connection, and ensuring every minute earns its place. Listening to Petra Wille and Teresa Torres map out the speaker lineup, workshops, and structural shifts, I found myself nodding along—this is the kind of thoughtful curation we need if we want product teams and product leaders to walk away with practical value, not just inspiration.
Listen to this episode on: Spotify | Apple Podcasts
What stood out immediately is the bold move to a single-track conference for 2026. In an era of gen ai hype and endless breakouts, this choice signals clear intent: tighter curation, a shared experience, and less FOMO. The team isn’t carving out a separate AI track—and I love that decision. Their stance is simple and sensible: No AI track—AI will show up everywhere, but not as a siloed topic. The team sees it as part of the everyday toolkit. That mirrors how high-performing, empowered product teams actually work today—AI Strategy and AI workflows are part of the operating system, not a side show.
The keynote lineup is already compelling. Christian Idiodi (SVPG) brings storytelling that turns product principles into habits you can actually use on Monday. Elaine Kasket, cyber-psychologist, exploring digital afterlife and AI replicas, will push us to think more deeply about the human side of our systems. And Teresa Torres will be sharing what she’s learning about AI—exactly the kind of continuous discovery mindset we need as we integrate LLMs into product discovery and delivery.
I’m also thrilled to see roundtables become what they’re calling an “alternative track.” That’s a smart way to deepen learning without fragmenting attention. The best conference ROI I’ve had often comes from targeted small-group conversations—where product trios compare approaches, swap metrics frameworks, or challenge each other’s product strategy assumptions. It’s a design choice that rewards curiosity and builds communities of practice.
We also get a behind-the-scenes look at Teresa’s Maker Studio workshop, where participants will build personal AI workflows. That’s exactly the hands-on, practitioner-first approach teams need right now—less demo theater, more systems that stick. If your roadmap includes integrating LLMs into continuous discovery or augmenting your team’s decision velocity, this kind of guided practice is gold.
The broader workshop slate looks deep and balanced. Expect returning favorites and practical frameworks: Rich Mironov on the realities of product leadership in complex orgs; Büşra’s metrics workshop translating outcomes into action; and an overview of additional workshops from Rich Mironov, Büşra Coşkuner, Marcus Castenfors, and Özlem Yüce. From success metrics to toolkits for product managers, the content spans IC to product management leadership—ideal if you’re stepping into new roles or scaling empowered product teams.
One of the most exciting evolutions is the Product Leadership Event, now a 1.5-day retreat. The format blends talk sessions, mini-workshops, dinners, and small-group excursions (boat rides, improv, etc.), giving leaders time and space to exchange playbooks, stress-test decisions, and build real relationships. It’s capped at 60 attendees (all in product leadership roles) to keep it intimate and useful. As someone who believes in outcomes vs output OKRs and first principles decision making, I appreciate how this structure encourages depth over breadth—and real accountability among peers.
Here are the core takeaways I’m carrying into my own planning: single-track means tighter curation, so every talk has to earn its place. Roundtables are growing into an “alternative track,” offering more ways to engage beyond stage talks. Workshops go deep and meet you where you are—IC, manager, or executive. And the leadership retreat expands to maximize learning from peers, not just from the stage. If you care about product discovery, product strategy, and conference networking that leads to actual business impact, this program looks thoughtfully engineered.
If you’re planning your 2026 calendar—or just curious how conferences evolve alongside the craft—this is a thoughtful walkthrough of what to expect. Come say hi to Teresa and Petra—on stage, at a roundtable, or somewhere in the hallway conversations that make these events memorable.
For more context and resources mentioned, explore: Product at Heart, Arne Kittler, Mind the Product, Christian Idiodi of Silicon Valley Product Group, Elaine Kasket, House of Beautiful Business, The 7 Habits of Highly Effective People by Stephen Covey, Rich Mironov, Marty Cagan, Claude Code, Codex by OpenAI, Marcus Castenfors, Büşra Coşkuner and her Success Metrics: A Playbook for Product Managers, Özlem Yüce’s Essential Toolkit for Product Managers, Petra’s Product Leadership Wheel (PLwheel), and Netlight.
Follow Teresa Torres: https://ProductTalk.org
Follow Petra Wille: https://Petra-Wille.com
Full transcripts are only available for paid subscribers.
I’ve led product teams through countless discovery cycles, and nothing has accelerated our learning loops like AI. By weaving AI into our continuous discovery practice at HighLevel, I cut time-to-insight, reduce risk earlier, and keep our product strategy relentlessly focused on customer outcomes.
AI streamlines product discovery by accelerating research, prototyping, and validation, enabling teams to make faster, smarter, and user-driven decisions.
In the research phase, I use gen ai and LLMs for product managers to synthesize interviews, cluster themes, and surface unmet needs in minutes instead of days. Pairing those qualitative insights with behavioral signals in Amplitude analytics helps me spot high-intent cohorts and friction points at scale, so our problem framing is both human-centered and data-backed.
From there, I translate insights into crisp hypotheses and prioritize with the Kano Model and outcomes vs output OKRs. To keep experiments honest, I define a minimum detectable effect (MDE) up front and design A/B testing plans that reflect realistic traffic and seasonality, ensuring our decisions are statistically grounded rather than anecdotal.
Prototyping is where gen ai for product prototyping really shines. I spin up multiple UX flows, UI copy variants, and edge-case scenarios using prompt engineering, then iterate with rapid feedback from product trios. When needed, I mock in-app guides and product tours to validate onboarding concepts before we commit to code, preserving velocity without sacrificing quality.
For validation, I lean on a mix of lightweight experiments—fake-door tests, concierge pilots, and targeted A/B testing—augmented by in-product surveys via Pendo or Intercom. For AI-powered features, I apply eval-driven development to measure relevance, latency, and safety, so we can ship responsibly while maintaining the pace of learning.
This approach only works when the team is structured to move fast. Empowered product teams and product trios own discovery end-to-end, with clear guardrails around data governance, privacy-by-design, and AI risk management. That alignment lets us shift from opinions to evidence, and from output to outcomes, without friction.
If you’re getting started, pick one discovery loop to transform: automate research synthesis, prototype two to three variants with AI, and validate with a tightly scoped experiment. Instrument your analytics, track time-to-insight and time-to-prototype, and iterate your product roadmapping and sprint planning with what you learn. The payoff is immediate: faster cycles, stronger conviction, and a more user-driven path to product-led growth.
I set out to create a lightweight, high-impact “Pendo Wrapped” experience for our users—and I did it in under 10 minutes with Pendo MCP. As a VP of Product Management, I’m constantly looking for fast, pragmatic ways to turn product insights into moments that drive engagement. This experiment was about transforming raw analytics into a concise, celebratory year‑in‑review that motivates customers to explore more value.
When I say “Pendo Wrapped,” I mean a simple, narrative-style summary of usage highlights: what got adopted, which moments mattered, and where value showed up most clearly. Framed well, that story reinforces product‑led growth by reminding users why they chose us, nudging them toward the next best action, and strengthening activation and retention without heavy development work.
My approach was straightforward: define a clear objective (celebrate milestones and prompt the next step), choose a focused set of metrics (adoption, engagement, and activation), and target relevant segments. Then I layered the narrative on top of existing analytics using in‑app guides and product tours to deliver the experience where it matters most—inside the product.
The reason it took minutes, not hours, is that Pendo MCP let me work with what we already had—segments, saved reports, and proven guide templates—so I could spend time on the story, not the scaffolding. No code, minimal configuration, and a crisp call to action made it feel polished without being heavy.
Increase revenue, cut costs, and reduce risk with Pendo’s Software Experience Management platform. Optimize the entire software experience to drive adoption and improve engagement.
If you want to replicate this quickly, start by selecting one user segment and three metrics that matter to them, write a two‑sentence narrative that connects those metrics to outcomes, and ship a short in‑app guide with a single, purposeful CTA. That’s enough to deliver a personalized year‑in‑review feel and spark immediate exploration—no new infrastructure required.
What surprised me most was how a small, story‑driven touch created outsized alignment across customers and internal teams. It turned analytics into advocacy, reminded our users of the value they’re already getting, and opened the door to deeper adoption. If you’re pursuing product‑led growth, a fast “Pendo Wrapped” is one of the highest‑leverage experiments you can run this week.
Your tender inbox contains a deadline, a stack of attachments, and a chain of decisions that still lives in people’s heads. The tempting response is to buy a model that can read PDFs. That solves only the most visible part of the problem.
A useful tendering product must determine which documents matter, extract requirements with evidence, match those requirements to a catalog, retrieve approved pricing, draft an offer, identify uncertainty, and route exceptions before anything reaches the customer. If you lead AI or product strategy for a manufacturer or supplier, your first goal should not be an autonomous bidder. It should be the smallest tender category in which every decision can be observed, evaluated, and improved.
Pick a bounded quote, not the entire tendering department
Construction tendering is too broad for a credible first release. Product categories have different terminology, selection rules, catalogs, pricing structures, and exception patterns. A system that works for one bounded category has not automatically learned how to quote every building product.
One effective wedge started with radiator requests for a single design partner before expanding to other building products. That constraint made the catalog, expected outputs, and expert reviewers knowable. It also created a place to learn the real workflow rather than designing from an idealized process diagram.
Choose your wedge using operational criteria, not enthusiasm for the model:
The request appears often enough for reviewers to recognize recurring patterns.
The relevant catalog is bounded and maintained by a clear owner.
A domain expert can explain why a product is suitable, unsuitable, or uncertain.
The correct price can be traced to an approved system or document.
Historical tenders and reviewer corrections are available for evaluation.
An error can be caught during review before it becomes a customer-facing commitment.
A design partner is especially valuable because the work is not fully documented. In one implementation, the product team spent a week observing the process on-site. That kind of observation exposes the browser tabs, informal checks, catalog shortcuts, and exception handling that an interview alone can miss.
Follow one tender from the incoming email to the final offer. At every handoff, record five things: the input, the decision, the evidence used, the person or system responsible, and the condition that triggers an exception. If you cannot state those five things, you do not yet have a well-defined agent task.
Keep three layers of information separate from the beginning:
Stated requirements: what the tender explicitly asks for, with the originating file, page, section, or table cell.
Interpretations: conclusions the system or reviewer draws when terminology is ambiguous, incomplete, or inconsistent.
Commercial decisions: the selected product, approved price, assumptions, exclusions, and offer language.
This separation matters because a polished offer can hide a weak inference. A reviewer needs to see where the tender ends and the system’s judgment begins.
Define the first outcome as a review-ready tender case: organized source documents, structured requirements, proposed product matches, price provenance, unresolved issues, and a draft offer. That is a more useful product boundary than “understands construction PDFs.” It gives the reviewer something concrete to accept, correct, or reject.
Turn the workflow into a decision graph with specialist agents
A chatbot is the wrong mental model. Tendering is a decision graph in which an early classification or extraction error can contaminate every downstream step. Real packages can range from a short request to more than 1,800 pages describing an entire building. The system therefore needs to plan work, retain state, reconcile evidence, and know when coverage is incomplete.
Use agents only where a task requires interpretation, planning, or exception handling. Keep exact operations – file handling, arithmetic, catalog queries, identifiers, template validation, and access control – in deterministic code or approved systems.
Stage
Preferred control
Required output
Intake
Rules plus a classifier
A tender case containing the email, attachments, document types, and routing status
Requirement extraction
Parser tools plus a specialist agent
Structured requirements with source locations, missing fields, and ambiguities
Product matching
Catalog retrieval plus a reasoning agent
Candidate products, requirement coverage, incompatibilities, and rationale
Pricing
Approved database, CPQ, or pricing service
Exact product-price records with source and validity information
Offer generation
Controlled template plus a drafting agent
A draft that distinguishes confirmed facts, assumptions, and exclusions
Quality review
Rules plus a separate review agent
A pass or block decision with issue codes and supporting evidence
Human approval
Domain and commercial policy
An approval, correction, rejection, or escalation that becomes evaluation data
Each agent should have an explicit contract. Specify its permitted inputs, tools, output schema, evidence requirements, completion test, and escalation behavior. “Find the right product” is not a contract. “Return catalog candidates that meet the extracted requirements, identify uncovered requirements, cite the catalog evidence, and abstain when no candidate qualifies” is much closer to one.
For large document sets, require the workflow to maintain a task plan. It should inventory files, identify relevant sections, process bounded units of work, track completed and pending units, reconcile repeated or conflicting requirements, and run a final coverage check. A generated answer is not proof that the package was fully processed.
The review agent deserves its own role. Asking the drafting agent to “check your work” keeps creation and approval inside the same reasoning path. A separate reviewer can inspect the draft against the extracted requirements, catalog evidence, price records, and policy rules. It should return defects and a gate decision rather than silently rewriting the offer. Silent rewriting makes it harder to identify which upstream component failed.
This pattern has practical value because a dedicated review agent can catch errors before human review, much like a separate code review step. Independence comes from the reviewer’s task and evidence contract; adding more agent personas without distinct responsibilities only creates orchestration overhead.
The interface is part of the architecture. During discovery, a dedicated web workbench can be more useful than hiding the workflow behind a legacy integration. Put the source document, extracted requirement, proposed match, price evidence, and review issue within the same review path. That gives the product team control over feedback capture and makes the reason for each correction visible. One tendering product used its own web application to iterate toward greater automation rather than beginning as a backend-only integration.
You can still read from and write to existing systems at defined boundaries. The distinction is between integration and dependence: integrate with systems of record for catalogs, prices, customers, and approved quotes, but do not let an inflexible legacy screen determine how reviewers inspect an emerging AI workflow.
Evaluate every decision before judging the complete quote
An end-to-end result tells you whether a tender case failed. It rarely tells you why. If the final product is wrong, the defect may have come from document routing, requirement extraction, catalog retrieval, product reasoning, pricing, drafting, or review. A single overall accuracy score collapses those failure modes into an unactionable number.
Pricing: exact agreement with the approved source, correct product-price association, formula validation, and rejection of unavailable or invalid records.
Offer generation: required-field completeness, consistency with selected products and prices, correct treatment of assumptions, and unsupported statements.
Review: detection of known defects, false blocks on valid cases, issue classification, and evidence quality.
Slice failures by characteristics that change the work: document length, file type, layout, product category, presence of tables, and conflicting or revised requirements. An aggregate score can improve while performance deteriorates on the long or unusual tenders that consume most reviewer attention.
Use end-to-end measures for the product outcome: review time, correction volume, correction severity, exception rate, percentage of cases that reach the defined completion state, and whether the workflow finishes before its operational deadline. Keep commercial outcomes separate from model correctness. Quote acceptance or win rate can be affected by price, availability, competition, customer relationships, and sales execution; it should not be treated as a clean extraction or matching metric.
Observability must connect those layers. For each tender case, retain the task plan, agent inputs and outputs, tool calls, retrieved catalog or price records, prompt and model versions, gate decisions, latency, failures, and human corrections. Complex agent chains can exceed what generic monitoring exposes, which is why custom tracing and Agent Analytics became necessary in a production tendering workflow.
Capture reviewer feedback as structured data, not only as an edited final document. Store the original output, corrected value, responsible stage, reason code, evidence used, and final disposition. Useful reason codes include missing requirement, incorrect extraction, unsupported product match, pricing issue, unresolved conflict, invalid assumption, and drafting defect.
Do not feed every edit directly back into the system and call it self-learning. A reviewer may change wording for preference, apply customer-specific knowledge, or correct an upstream error in the final draft. Validate the correction, assign it to the right component, and add it to the corresponding evaluation set. That turns human review into controlled learning rather than an untraceable feedback loop.
Release changes through two gates. First, the modified agent must pass its own evaluation set. Second, the complete workflow must pass the end-to-end set because an improvement in one component can change the assumptions of another. The trace should show exactly which version produced every customer-facing artifact.
Earn autonomy one commercial boundary at a time
“Straight-through processing” is incomplete unless you define where the straight-through path ends. Automatically extracting requirements is not the same risk as automatically selecting a product, committing a price, writing to the CPQ, or sending an offer to a customer.
Use an autonomy ladder with an explicit boundary at each stage:
Shadow: the system processes live-shaped cases, but its outputs do not affect the operational tender.
Assist: it organizes documents and extracts requirements while a person performs matching, pricing, and drafting.
Draft: it proposes products and produces an offer, but a human must review and approve every case.
Gated processing: it completes predefined internal actions for in-scope cases and sends all exceptions to a reviewer.
External dispatch: it sends an offer without case-by-case approval only when commercial policy explicitly permits that action and every required gate passes.
Eligibility for a higher-autonomy path should be machine-checkable. At minimum, confirm that the product category is in scope, every required document was processed, required fields are present, source evidence is attached, conflicts and revisions are resolved, the product match satisfies its evidence rules, the price comes from an approved valid source, the review agent reports no blocking defect, and the full trace is retained.
A wrong product or price can create margin, delivery, contractual, and customer-trust exposure. If an offer may create a binding commitment, keep human approval until the appropriate commercial and legal owners have defined the policy for automatic dispatch. The safe alternative is to automate preparation while preserving approval at the commitment boundary.
Operational controls matter after launch. Give reviewers a visible exception queue, make the reason for every block legible, preserve manual processing when the AI path is unavailable, and provide a way to suspend autonomous actions without disabling access to already processed cases. Assign owners for catalog quality, pricing validity, tender policy, model behavior, and production reliability; otherwise each exception will bounce between teams.
Expansion should follow evidence and customer pull. A request to replace an existing CPQ system is a meaningful product-market signal, but it is also a change in product scope. CPQ replacement introduces responsibilities for quote versions, approval policy, catalog administration, pricing governance, integrations, and records. Treat that request as a roadmap decision, not as proof that the original agent workflow already covers those capabilities.
Key takeaways
Start with one product category, one known workflow, and reviewers who can explain the correct decision.
Optimize first for a review-ready tender case, not an impressive answer from a general chatbot.
Use deterministic systems for exact operations and specialist agents for interpretation, planning, and exceptions.
Require structured outputs, source evidence, explicit completion tests, and an abstain-or-escalate path from every agent.
Evaluate each stage separately, then use end-to-end metrics to measure the operational outcome.
Increase autonomy only when observable eligibility gates protect the next commercial boundary.
At your next roadmap review, put one representative tender on the screen and draw the decision path from email to offer. Name the owner, evidence, pass condition, and exception path for every node. Wherever those are missing, the next task is workflow discovery, not another agent. Once the graph is explicit, you can automate one bounded decision, measure it, and earn the right to automate the next.
Your AI feature has passed the demo. Customers want it, leadership wants a date, and the team believes the remaining risks can be handled before launch. The problem is that nobody can state what evidence would make the feature safe enough to release – or who can stop it when that evidence is missing.
This is where AI ethics has to become product governance. You need a repeatable way to classify risk, set release conditions, assign decision rights, test safeguards, and respond when production behavior differs from the demo. The goal is not to eliminate uncertainty. It is to make uncertainty visible and govern the consequences.
Start with a release contract, not a list of principles
For each AI capability, write a short release contract before implementation begins. It should answer:
What decision or task is the product helping with? Describe the user outcome, not the model output. Generating a response is an output; helping a support agent resolve a request accurately is an outcome.
What must the system never do? Name unacceptable behavior such as exposing restricted data, presenting unsupported claims as facts, acting without required confirmation, or concealing that AI influenced an outcome.
Who can be affected? Include people represented in the data, people discussed in generated content, employees asked to rely on the output, and anyone subject to a downstream decision.
How consequential is a wrong result? Separate an inconvenient suggestion from an output that can affect access, money, employment, safety, privacy, or another difficult-to-reverse outcome.
What evidence is required to ship? Tie every material risk to an evaluation, control, review, or operational test. Avoid release criteria such as reasonable quality or adequate safeguards; two reviewers can interpret those phrases differently.
What will stop or reverse the feature? Define the conditions for disabling an action, reverting a version, narrowing availability, or returning the workflow to human handling.
Treat these conditions as part of the acceptance criteria. If a trust condition fails, the feature has not passed release readiness even when its primary quality metric looks strong. That keeps ethical constraints from becoming optional work negotiated away at the end of the schedule.
Classify the use case by consequence, autonomy, and reversibility
A model does not have one fixed risk level. The same underlying model can draft a headline, recommend an account action, or execute that action. Governance should therefore follow the use case rather than the model name.
A practical classification starts with three questions:
Consequence: What happens if the output is wrong, biased, misleading, or disclosed to the wrong person?
Autonomy: Does the system inform a person, recommend a decision, or take the action itself?
Reversibility: Can the affected person notice the result, challenge it, and restore the prior state without disproportionate effort?
Use those answers to choose a product path. A reviewable drafting aid may rely on disclosure, editing controls, standard evaluations, and ordinary monitoring. A consequential recommendation needs stronger evidence, an accountable human reviewer, and a clear appeal or correction path. An autonomous, hard-to-reverse action should not launch until the team can justify the autonomy, constrain permissions, require confirmation where appropriate, and demonstrate a reliable override.
Do not confuse a human in the workflow with meaningful human oversight. A person who lacks context, time, authority, or a usable way to reject the output is functioning as a rubber stamp. For higher-risk actions, the reviewer needs the evidence behind the recommendation, a clear indication of uncertainty or limitations, and the authority to choose a non-AI path.
Record the classification in an AI risk register. Each entry should contain the risk scenario, affected parties, possible impact, warning signals, preventive control, detection method, response, owner, required evidence, residual risk, and the person authorized to accept that residual risk. A model defect belongs in the backlog; a plausible future failure belongs in the risk register; a failure already affecting users belongs in incident management. Keeping those states distinct prevents serious risks from disappearing into a generic bug queue.
Likelihood will often be uncertain before production. Do not turn that uncertainty into a convenient low-risk label. Record what is unknown, how the team will test it, and which production signal will cause a review. For a consequential or difficult-to-reverse feature, I would also separate the person implementing the control from the person accepting the remaining risk.
Turn governance into four evidence-based release gates
A governance meeting should inspect evidence, not collect reassuring opinions. Four gates cover the path from data collection to production response. The depth of each gate should match the use-case classification.
Data gate: prove that the inputs are governed
Trust problems often begin before a prompt reaches the model. The data gate should make the full path of customer and organizational data inspectable.
Document what data is collected, where it came from, why it is needed, and which product purpose it serves.
Identify the applicable basis for processing and make consent flows explicit where consent is used. Legal requirements depend on the product, data, and jurisdiction, so product teams should validate this with qualified privacy and legal partners rather than infer an answer from a generic checklist.
Remove fields that are not needed for the stated outcome. Data minimization reduces both privacy exposure and the number of inputs that can produce unexpected behavior.
Map data lineage across ingestion, retrieval, model calls, logs, analytics, support tools, and vendors. A deletion promise is not credible if the team cannot locate every copy.
Apply role-based access to raw inputs, retrieved context, generated outputs, and operational logs. Access to the application should not automatically imply access to all AI interaction data.
Set retention and deletion rules, then test that they work across the full data path rather than only in the primary database.
The gate passes when the team can trace an input, explain its permitted use, name who can access it, and show how it is removed. A policy document without an enforceable data path is not sufficient evidence.
Model gate: test the failures that matter to the use case
Map every important risk in the register to an evaluation. If a risk has no test, state which manual review or production control provides the evidence instead.
Define the passing condition before reviewing final results. Moving a threshold after seeing a disappointing result turns a gate into a negotiation.
Test normal requests, ambiguous requests, edge cases, adversarial prompts, and realistic multi-step interactions. A polished set of happy-path prompts will not expose operational failure modes.
Compare performance across the user groups and contexts relevant to the product. Aggregate quality can conceal a meaningful gap affecting a smaller group.
Red-team prompts, retrieved context, tool use, and permission boundaries. For an agentic workflow, the safety of the text is only one part of the problem; the allowed action is another.
Keep the evaluation set and results tied to the model, prompt, retrieval configuration, tools, and policy version that produced them. Otherwise, a passing report can outlive the system it evaluated.
When an LLM must answer from known organizational information, a retrieval-first pipeline can ground the response in authoritative material. It does not remove the need for evaluation. Test missing documents, conflicting documents, stale content, access-restricted content, and questions the knowledge base cannot answer. The safe behavior may be to abstain, ask for clarification, or route the task to a person.
Experience gate: help users exercise judgment and control
Disclosure is useful only when it changes what a person can understand or do. Place it near the AI-assisted decision, in plain language, and explain the limitation that matters in that moment. A broad statement hidden in terms and conditions does not help a user assess a specific output.
Make it clear when AI generated, transformed, recommended, or acted on information.
Let users inspect, edit, reject, or correct an output before a consequential action where that control is meaningful.
Separate generated content from verified facts in the interface. Do not use confident UX writing to imply certainty the system cannot support.
Explain what data the feature needs and what changes when the user turns it off.
Provide a non-AI or human-assisted path when the AI path is unsuitable for the task.
Test whether users understand the system’s role. A control that exists but cannot be found or understood is not an effective safeguard.
Match the amount of friction to the consequence. Requiring confirmation for every low-impact suggestion can train users to click through automatically. For a high-impact or hard-to-reverse action, the extra pause may be the safeguard that preserves meaningful control.
Operations gate: demonstrate that failure can be contained
Pre-launch evaluations cannot cover every production context. The operations gate determines whether the team can detect, contain, and learn from behavior that escaped testing.
Monitor model behavior and customer impact. Technical availability can look healthy while unsupported outputs, harmful actions, or repeated user corrections are increasing.
Assign an owner and response for each alert. An unowned dashboard is visibility without control.
Create a kill switch or permission cutoff for risky actions, plus a rollback path for model, prompt, retrieval, and tool changes.
Test the rollback under realistic access and dependency conditions. A safeguard that nobody has exercised may fail during the incident it was meant to contain.
Prepare an incident playbook covering triage, containment, evidence preservation, affected-user assessment, communication, recovery, and the decision to restore service.
Keep a human override for high-risk actions and verify that the operator can use it without depending on the failing AI path.
This gate passes when the team can answer three questions without improvising: How will the failure be detected? Who can stop it? What evidence is required before it is turned back on?
Assign decision rights across the product lifecycle
Governance slows teams when everyone can raise concerns but nobody knows who decides. Put decision rights beside the risk register and release gates.
Product: owns the intended outcome, use-case classification, release contract, customer trade-offs, and completeness of the risk register.
Engineering and data: produce evidence for system behavior, data lineage, access controls, evaluations, technical constraints, and remediation.
Design and research: verify disclosure, comprehension, correction, appeal, and user control in the actual workflow.
Security and privacy: examine access, abuse paths, data handling, vendor exposure, and response controls.
Legal and compliance: interpret applicable obligations and identify where a product decision creates legal exposure. Product leaders should bring these partners in while choices are still reversible.
SRE and operations: own observability, alerting, rollback mechanics, incident readiness, and production recovery with the product team.
Executive risk owner: accepts material residual risk when the decision exceeds the product team’s authority and ensures that the required mitigation has resources.
The review itself should be a decision forum, not a status meeting. Send the release contract, risk register, failed and passed evaluations, unresolved questions, and requested decision in advance. End with one of four outcomes: approved, approved with explicit conditions, returned for more evidence, or rejected. Record the rationale and the event that will trigger another review.
Apply the same discipline to purchased models and AI services. A vendor can operate part of the stack, but it cannot absorb your accountability to customers. Due diligence should cover model provenance, data use and retention, access, evaluation evidence, incident history, change notification, and subcontracted dependencies. Contracts should carry operational commitments such as service levels, deletion obligations, audit rights, and incident responsibilities into the vendor relationship.
If a vendor cannot answer a material question, record the item as unknown. Do not silently translate missing evidence into low risk. Decide whether a compensating control – limited data, narrower permissions, independent evaluation, or a manual workflow – makes the unknown acceptable. If not, change the design or supplier.
Treat launch approval as a monitored, reversible decision
Approval should attach to a defined system configuration and use case, not to the feature name forever. A model change, system-prompt change, new retrieval corpus, broader user group, expanded data access, new tool permission, or shift from recommendation to autonomous action can invalidate earlier evidence. Put those change triggers in the original approval.
Launch with the smallest exposure that can produce useful operational evidence. Watch model-quality signals alongside user corrections, overrides, complaints, unexpected actions, access violations, and downstream customer impact. Set an owner and response for each signal before rollout. Waiting for a broad satisfaction metric to move can leave a concentrated harm hidden inside an apparently successful launch.
Customer trust also depends on what you reveal outside the internal review. A customer-facing trust center can publish the AI system’s role, material limitations, relevant data practices, available controls, change history, and a path for reporting problems. Model facts, limitations, and change logs make responsible operation visible. Candor about a boundary is more useful than a vague claim that the system is responsible or safe.
Key takeaways
Govern the use case, not the model in isolation. Consequence, autonomy, and reversibility determine the controls you need.
Pair every success metric with an unacceptable outcome and observable release condition.
Use one living risk register to connect risk scenarios, evidence, owners, safeguards, residual risk, and review triggers.
Require evidence across data, model behavior, user experience, and production operations before release.
Treat human oversight as a designed capability. The reviewer needs context, time, authority, and a usable alternative.
Carry governance into vendor selection, contracts, monitoring, incident response, and material system changes.
Take one AI item from your current roadmap and write its release contract before the next planning or governance meeting. Name the intended decision, unacceptable outcomes, affected people, required evidence, stop conditions, and accountable risk owner. Any blank you cannot fill is not paperwork still to complete. It is product work you have found before customers find it for you.
Happy New Year! I’m kicking off 2026 with a behind-the-scenes look at what’s changing in my product practice, the experiments I’m running with my teams at HighLevel, and the trends I’m most energized by—especially around continuous discovery, AI workflows, and building stronger coaching cultures.
If you want to listen to the conversation that sparked many of these reflections, you can find it here: Spotify | Apple Podcasts.
Why Teresa sunset the live deep-dive cohorts—and how on-demand and the new Discovery Habits Toolbox better support real behavior change. This pivot resonated with my own experience: some skills, especially discovery habits, only stick when they’re reinforced in the flow of real product work, not just in a time-boxed cohort. In my org, we’re leaning into on-demand learning paired with manager coaching to drive durable behavior change.
What leaders actually need to coach interviewing, assumption testing, and core discovery habits inside their orgs. I’ve found that empowered product teams thrive when leaders have lightweight coaching tools, practical prompts, and clear expectations for product trios. This is less about one-off training and more about building communities of practice where deliberate practice and feedback loops become routine.
Why training is shifting toward ongoing, leader-supported learning (and how AI will accelerate the shift). AI Strategy isn’t just about tools—it’s about learning systems. For LLMs for product managers to create leverage, we need eval-driven development, privacy-by-design, and clear guardrails. I’m building AI workflows that enable managers to review interviews, spot anti-patterns, and nudge teams toward better decisions—without replacing critical thinking.
Teresa’s move into paid subscriptions and why AI content doesn’t fit the classic “design once, run for years” course model. I see the same reality in my content roadmap: the half-life of AI guidance is short. That pushes us toward subscription models, tighter feedback loops, and a more adaptive go-to-market strategy for education products.
A sneak peek into the AI tools Teresa is building for discovery work—from interview coaching to near-ready interview snapshot generation. I’m particularly excited by tooling that scaffolds better interviews, sharpens assumption testing, and speeds up synthesis without skipping the human judgment step. These capabilities map directly to where I want my teams investing time: spending less energy on admin and more on learning from customers.
Petra’s plans for the year: community building with Product at Heart, a new product leadership email course, her Product Leadership Wheel, and workshops launching in Cairo. As someone who believes in conferences as high-quality “energy wells,” I’m inspired by how these programs create momentum for leaders who are upgrading their coaching muscles.
The role of conferences and retreats in staying grounded, inspired, and connected. I treat these gatherings as strategic resets—spaces to test ideas, confront blind spots, and deepen my network for future collaboration. The best outcomes often come from serendipitous hallway conversations and hands-on sessions where you can pressure test frameworks with peers.
How Teresa is staying on top of academic research (and why “synthetic users” aren’t ready for prime time). I agree: while synthetic data can be useful for scaffolding, it’s not a substitute for direct customer contact. Combine academic rigor with real-world interviewing and strong data governance—especially when operating under General Data Protection Regulation (GDPR).
The shared challenge of evaluating vendors and conference speakers making questionable AI claims. My heuristic: ask for clear problem statements, reproducible evaluations, grounded benchmarks, and a path to safe deployment. If a pitch can’t show measurable uplift or ignores compliance, it’s not ready for empowered product teams.
Key takeaways I’m carrying into 2026: delivery models matter; leaders need coaching tools, not just training; AI is reshaping how we teach and learn; experimentation is the theme of 2026; and community still energizes. That’s the blueprint I’m using to strengthen continuous discovery, refine our AI workflows, and sustain high standards in product management leadership.
What about you? How are you integrating AI workflows into your discovery practice, and what coaching tools are helping your managers reinforce the right habits? Share your approach—I’d love to learn what’s working in your context.
Resources & Links:
Follow Teresa Torres: https://ProductTalk.org
Follow Petra Wille: https://Petra-Wille.com
Teresa’s website: Product Talk
General Data Protection Regulation (GDPR)
Product Talk Academy
Deliberate Practice – ATP episode where Teresa talked about the ending live cohorts for Deep Dive classes
