I build products under constant pressure to learn faster without breaking trust. Claude Code has become a pragmatic addition to my AI product toolbox because it helps me move from idea to evidence with less friction—while keeping engineering, design, and compliance in the loop.
“Claude Code for Product Managers explained: what it is, why it matters, and how it helps PMs prototype, validate, and move faster.” That line captures the essence. In practice, I use it to turn ambiguous problem statements into tangible artifacts—API stubs, SQL queries, test data, and lightweight prototypes—that sharpen conversation and accelerate decision cycles.
What is it in PM terms? A code-aware assistant that helps me prototype safely and quickly. I can generate example API calls, transform messy CSVs for retention analysis, draft instrumentation plans for Amplitude analytics, or spin up a mock service to validate an integration. Because it understands structure, it’s effective at scaffolding small utilities (e.g., a data cleaner or a CLI harness) that make discovery and validation faster.
Day to day, Claude Code reduces handoffs. If I’m exploring a new partner integration, I’ll have it produce a curl library and a Postman collection, then annotate each step with acceptance criteria and expected responses. When I’m shaping a feature, I lean on it to outline event taxonomies and feature flags so that engineering can wire telemetry without guesswork. For insights work, I’ll ask it to propose SQL for cohort, funnel, and retention analysis—always verifying against source schemas before anything touches production.
Speed is only useful when it improves signal quality. I anchor the workflow in continuous discovery: small hypotheses, thin-slice prototypes, and fast instrumentation. Claude Code helps me estimate A/B testing readiness (including minimum detectable effect), generate smoke tests for critical user paths, and structure an eval-driven development loop so we learn from every iteration. It also supports context window management by summarizing long PRDs into the few constraints a prototype must respect.
Governance matters. I apply AI readiness and AI risk management principles: never paste secrets or PII, isolate sandboxes, and log prompts as docs-as-code for auditability. I prefer a retrieval-first pipeline that feeds approved product docs, OpenAPI specs, and design tokens so generations stay grounded. When tools are integrated, I favor the Model Context Protocol (MCP) to constrain capabilities and maintain least-privilege access. Human-in-the-loop review is non-negotiable—especially for anything that might influence customer data or pricing.
The best outcomes show up in product trios. I’ll facilitate a live session with design and engineering: we co-create prompts, compare alternatives, and converge on a thin slice we can ship. That collaboration keeps us empowered, reduces interpretation drift, and turns Claude Code into an accelerant rather than a sidecar. Over time, the trio curates a reusable prompt library for PRD outlines, experiment checklists, and integration playbooks.
Getting started is straightforward: define a safe environment, assemble your authoritative corpus (requirements, specs, taxonomies), and codify a few high-value templates—API exploration, instrumentation plans, sandbox data generators, and acceptance tests. Track impact with simple, objective metrics: cycle time from hypothesis to instrumented prototype, time-to-first-signal, and the proportion of decisions made with data versus opinion.
There are pitfalls. Hallucinated fields can creep into API calls, schema drift can break generated queries, and “clever” refactors may miss edge cases. I mitigate this by grounding generations in current specs, asking for unit tests alongside any code, and validating against a staging environment before anyone talks about production. Treat Claude Code as a collaborator, not an oracle.
If your mandate is to learn faster, de-risk bets, and ship with confidence, Claude Code is worth adopting. Used thoughtfully, it compresses the distance between questions and answers, elevates product discovery, and lets teams validate more ideas with fewer meetings—without compromising on governance or quality.
In the age of AI, I’ve come to believe we’re all builders—yet not all building is the same. There is a very meaningful difference between building to learn (known as product discovery) versus building to earn (known as product delivery). When we confuse the two, we waste precious time, budget, and team energy on output over outcomes. My goal in this FAQ-style reflection is to clarify when and how to choose each mode so we can make smarter, faster, more confident product decisions.
Why does this distinction matter so much right now? Because as the cost of product delivery continues to drop, the scarce resource shifts from shipping capacity to clarity of problem, solution, and value. Cloud infrastructure, CI/CD, feature flags, and even gen AI code assistance have made it cheaper to launch. That’s great—but if we don’t learn the right things before we scale, we’ll efficiently deliver the wrong product. Discovery is how we de-risk that.
What do I mean by build to learn? I use discovery to quickly validate problems, test value, and shape solutions before committing delivery teams to scale. In practice, that means continuous discovery with customer interviews, rapid prototyping, and lightweight experiments that put us in front of real users fast. I rely on product trios and empowered product teams to co-own outcomes, not just output, and I anchor decisions with outcomes vs output OKRs so we stay focused on measurable impact.
How do I structure discovery sprints? I start with an opportunity solution tree to map customer pain points and candidate solutions, then select the smallest test that can invalidate a risky assumption. When signals are ambiguous, I refine the questions and instrument better learning loops rather than pushing harder on delivery. For experiments, I keep a bias to speed: clickable prototypes, concierge tests, or gen ai for product prototyping often reveal more in days than a coded MVP does in weeks. When experiments go live, I use a clear minimum detectable effect (MDE) and resist reading noise as signal.
Where does AI change the calculus? LLMs for product managers are turbocharging discovery by accelerating research synthesis, persona drafts, and early concept validation. I pair that with eval-driven development to set crisp acceptance criteria for AI behaviors before any production integration. Prompt engineering and conversation design are part of the toolkit, but the same rule applies: prototype to learn, not to impress. AI can make bad ideas cheaper to build—so disciplined discovery matters more than ever.
So when do I switch to build to earn? Once I have evidence of value and feasibility, I shift into product delivery to scale with quality, security, and reliability. This is where I bring in product roadmapping and sprint planning, DORA metrics to monitor deployment frequency and lead time, and strong SRE and observability practices to safeguard the user experience. The handoff isn’t a wall; discovery continues inside delivery to refine scope, reduce risk, and maintain momentum.
What pitfalls do I watch for? The biggest is treating delivery as discovery—shipping features to “see what happens” without a clear learning thesis. Another is tech-first decisions driven by technology FOMO instead of product strategy and customer value. I also see teams set output-based commitments that crowd out learning; outcomes vs output OKRs keep us honest. And when considering build vs buy, I evaluate whether the capability differentiates us; if not, I’ll buy to preserve discovery capacity on what truly matters.
My operating conviction is simple: invest early and deliberately in build to learn so build to earn becomes high-confidence, high-velocity, and high-impact. In practical terms, that means smaller bets, faster feedback, clearer outcomes, and tighter collaboration across product, design, and engineering. If we get discovery right, delivery feels inevitable—and customers feel understood.
I move fastest in Generative AI when I strip work down to its essential signals. At HighLevel, I rely on a single-page format—”Prototyping Requirements: The One-Pager for AI PMs”—to turn ideas into testable artifacts within hours, not weeks. This approach reinforces AI Strategy, minimizes coordination overhead, and keeps Product Management focused on learning over ceremony.
“Prototyping requirements go rogue: one page, zero bureaucracy, built for AI. Shape concepts fast, prompt tools directly, and get to the truth sooner.”
In practice, my one-pager captures only what’s required to run an immediate experiment: the user problem, the target behavior change, success signals, core constraints, intended AI workflows, and the smallest realistic path to an evaluable demo. I also include example prompts, guardrails, and evaluation criteria so the team can apply prompt engineering and LLMs for product managers without guessing.
This is eval-driven development in action. I document a minimal hypothesis, concrete inputs/outputs, and a quick plan for metrics, including qualitative signals from product discovery and continuous discovery. By prompting tools directly, we expose assumptions early, shorten feedback loops, and build an AI product toolbox that compounds learning sprint after sprint.
I run this with a product trio to ensure we balance feasibility, usability, and value. We align on risks, dependencies, and what “good” looks like, then we integrate the learnings into product roadmapping and sprint planning. The result: fewer meetings, tighter collaboration, and empowered product teams delivering sharper outcomes with less friction.
If you want speed and clarity without sacrificing rigor, adopt the one-pager. It centers the conversation on evidence, accelerates AI workflows from prompt to prototype, and makes it obvious what to try next—and what to stop doing. Most importantly, it keeps the team focused on truth over theater, which is how great AI products actually ship.
I believe the future of product design isn’t about replacing designers—it’s about giving every team access to one. That’s why Banani grabbed my attention. It’s an AI product designer that doesn’t just generate code—it generates design. For solo founders, stretched design teams, and early-stage startups, that shift matters: it raises the design floor without lowering the creative ceiling.
I spent time with Vlad Solomakha (CEO & Co-founder), Vova Kovalchuk (CTO & Co-founder), and Vlad Ostapovats (Founding Growth) to unpack how they took Banani from a Figma plugin proof-of-concept to a canvas-first AI design tool generating hundreds of thousands of designs per week. Vlad brings a decade of design experience and a precise north star: AI should produce beautiful, tasteful design rather than average, undifferentiated UI.
The architectural choices stood out. They engineered their agent to handle parallel screen edits, manage per-screen context across canvases with hundreds of frames, and make surgical edits without regenerating entire screens. This is the kind of agentic AI work that product leaders have been waiting for: concrete advances in context window management, tool orchestration, and prompt engineering that translate into higher throughput without sacrificing quality.
Equally important is how they addressed the "gulf of specification"—the mismatch between how designers think visually and how agents understand text. Banani’s canvas-first approach acknowledges that design is spatial, hierarchical, and iterative. Rather than forcing a chat-first UX, they center the canvas and let the agent do production work while keeping the designer firmly in control. In practice, this narrows intent ambiguity, speeds up iteration, and preserves taste.
The team made another pivotal bet: Why Banani doesn’t compile running applications — just HTML/CSS mockups — and how that shapes everything. By decoupling the design artifact from runnable code, they optimize for velocity, taste, and exploration. In my experience, this separation is the right product strategy for early discovery and gen ai for product prototyping—move fast on aesthetics and flows, then converge on implementation once you’ve validated the direction.
I also appreciated their pragmatic evaluation approach. Instead of traditional evals, they spin up 10 screens from one prompt to compare models. It’s hands-on, outcome-based, and aligned with eval-driven development in real product environments. They’re relentlessly discerning about when to work around model limitations versus when to wait for the models to improve—an essential discipline when building at the edge of what’s possible.
Under the hood, context engineering and specialized agent tools do the heavy lifting. Per-screen history with shared project context enables precise, reversible changes across large canvases. The result: fewer destructive regenerations, more reliable design intent preservation, and a workflow that feels like collaborating with a strong mid-level designer who’s exceptionally fast and consistent.
If you want a quick tour, I recommend jumping to a few highlights: 20:13 Product Tour Canvas First AI, 33:40 Gulf of Specification, 42:54 Agent Architecture Under Hood, 48:48 State History Context Tricks, and 56:04 Navigating Busy Canvases. Each segment reveals a different layer of the system design and product thinking behind Banani’s canvas-first UX.
For product leaders, this is a compelling blueprint for raising the design floor while protecting the last mile of craft. It aligns with empowered product teams, continuous discovery, and LLMs for product managers who need leverage without losing judgment. If you’re exploring agentic AI in design, this is a thoughtful, execution-focused model worth studying and trialing on your next product tour or redesign.
Resources worth exploring: Banani and TL Draw. To hear the full conversation, you can listen on Spotify or Apple Podcasts. Then, pressure-test the approach inside your own product development lifecycle and see how a canvas-first AI designer reshapes your team’s velocity and quality bar.
Most MVPs take too long, cost too much, and still miss the mark. Over the past year, I’ve shifted my team to a prototyping prompts approach that lets us validate problem-solution fit in days, not months. The result is faster learning loops, clearer tradeoffs, and a dramatically higher hit rate on features that actually move the needle.
When I say prototyping prompts, I mean structured, layered instructions that guide gen ai systems to produce the right artifacts at the right fidelity. Instead of jumping straight to code, we generate concise problem briefs, user stories, interaction flows, low-fidelity UI descriptions, and test plans. Each pass is constrained by acceptance criteria and business outcomes, which keeps the work grounded in value rather than output.
Here’s the playbook my product trios use to go from idea to a testable MVP in 48–72 hours. First, we anchor on outcomes vs output OKRs and clarify the customer job-to-be-done using evidence from customer interviews and support data. This is classic continuous discovery, but we compress it by focusing on the single riskiest assumption to de-risk this week.
Second, we build a prompt scaffold. We specify the role, constraints, target users, success metrics, and the exact output format we expect. We also define evaluation upfront, borrowing from eval-driven development. For example, before any generation, we list the acceptance tests that a good solution must pass, including edge cases and compliance considerations. This discipline keeps hallucinations in check and improves repeatability.
Third, we spin up multiple prototypes in parallel. One prompt generates a lean product brief; another outlines user flows; a third proposes UI states and error handling. If we’re exploring voice, we add prompt engineering for voice to script dialogs and repair strategies. For data-heavy features, we call out retrieval-first pipeline patterns so the model references source-of-truth data rather than guessing.
Fourth, we validate with real users using the lightest-weight experiment possible. Fake-door tests, concierge workflows, and guided click-throughs let us measure intent before we invest. Where we can, we run quick A/B testing and size the effort using minimum detectable effect (MDE) so we don’t over- or under-sample. The point isn’t perfection; it’s fast, directional signal to inform the next iteration.
Fifth, we instrument and ship behind feature flags. We track activation, task completion, and time-to-value from day one. On the delivery side, we watch DORA metrics and deployment frequency to ensure we’re learning continuously rather than batching big bets. This bridges discovery and delivery so roadmaps reflect real-world feedback, not assumptions.
One recent example: we needed to evaluate a voice AI agent for appointment scheduling. In 72 hours, prompts produced the problem brief, dialog flows, error recovery strategies, and a sandbox to simulate inbound requests across three user personas. We exposed a thin slice to a pilot cohort, captured call outcomes, and iterated the repair prompts twice before writing any production code. The pilot converted at a higher rate than our control flow and gave us the confidence to invest in full integration.
This approach only works if we treat governance as a first-class concern. We bake in privacy-by-design, clear data governance boundaries, and AI risk management from the start. Prompts include guardrails on personally identifiable information, explicit constraints on data use, and links to approved sources. We also maintain a prompt repository with versioning and automated evaluations so changes are observable and reversible.
Practically, strong prompt scaffolds share three traits. They’re specific about context and constraints, they define success in measurable terms, and they separate concerns by artifact type. I’ll often ask for three variants with different tradeoffs, then run a quick synthesis prompt that highlights points of parity and differentiation. This gives the team structured options rather than a single, brittle path.
If you’re starting from zero, begin with one high-leverage workflow. Write a crisp outcome statement, draft your acceptance tests, and create a prompt that outputs a one-page brief, three user flows, and the top five risks with mitigations. Validate with five users in 48 hours, then decide: double down, pivot, or park. Rinse and repeat, and your product roadmapping and sprint planning will shift from speculation to evidence.
The bottom line is simple. Prototyping prompts won’t replace product judgment, but they will accelerate it. By turning ideas into testable artifacts in hours, you minimize waste, maximize learning, and ship better MVPs—fast.
Your team can turn a behavioral anomaly into a polished prototype within hours rather than over weeks, yet still stall when it is time to choose a problem, approve a test, or act on the result. That is the central trap in AI-accelerated product development: producing artifacts faster does not automatically produce better decisions.
The useful unit of acceleration is the complete learning loop: detect a meaningful signal, frame the opportunity, explore distinct hypotheses, validate the riskiest assumptions, ship with controlled exposure, and use production evidence to decide what happens next. You need one operating model across that loop, not a collection of disconnected AI shortcuts.
Key takeaways
Optimize for time from signal to a decision backed by evidence, not the number of analyses, prototypes, or tickets generated.
Give every investigation an outcome contract: the customer behavior, target cohort, primary metric, guardrails, and decision that the work is intended to inform.
Use AI to create alternatives that represent different value hypotheses. More cosmetic variants usually create more review work without expanding what you can learn.
Carry the same cohort, metric definitions, hypothesis, and constraints from discovery into the production experiment. This prevents the handoff from silently changing the question.
Let agents act only where their permissions, thresholds, audit trail, and rollback path are explicit. Autonomy should expand with evidence, reversibility, and trust.
Design one loop from a product signal to a decision
Most teams first apply AI to individual tasks. An agent summarizes a dashboard. A model drafts a product requirements document. A design tool generates a flow. A coding assistant implements it. Each task becomes faster, but the work still waits between tasks because nobody has defined what evidence is sufficient, who can make the next decision, or what outcome the change should affect.
An agent that discovers more anomalies while the product trio reviews opportunities through the same overloaded process has created a longer inbox. The bottleneck has moved; it has not disappeared. The remedy is to treat a decision-ready hypothesis, rather than an AI-generated artifact, as the unit of product work.
A practical discovery loop has the following sequence:
Write the outcome contract. Name the customer behavior you want to change, the cohort in which it matters, the primary outcome metric, the metrics that must not deteriorate, and the decision this evidence will support.
Map the driver tree. Break the outcome into observable behavioral drivers. This gives the agent a bounded search space and prevents a broad metric movement from producing an equally broad list of possible features.
Issue an investigation brief. Tell the agent which definitions, segments, releases, and time comparisons it may use; which data it may access; what it should monitor; and whether it may only recommend or may also initiate an approved workflow.
Require an evidence packet. An anomaly should arrive with the affected cohort, direction and materiality of the movement, relevant timing, instrumentation checks, plausible alternative explanations, and the next question worth answering.
Record the decision. The product trio should accept, reject, defer, or refine the hypothesis and state why. That decision becomes context for the next investigation instead of disappearing into a meeting.
For an onboarding problem, the outcome contract might identify accounts attempting their first meaningful setup, define the activation behavior precisely, name downstream retention and support demand as guardrails, and authorize the agent to investigate friction without changing the customer experience. That is much more useful than asking AI to find onboarding insights. The broad request has no stopping condition and no decision attached to it.
The driver tree then narrows the investigation. Activation might depend on starting setup, completing required configuration, reaching an initial value-bearing action, and returning to use that value. The point is not to make the tree exhaustive. It is to show which behaviors could plausibly explain the outcome and which are observable in your product data.
This is where continuous agents can provide real leverage. They can monitor established metrics, inspect funnel and cohort movements, and surface material changes such as an activation decline in a valuable cohort or a retention change following a release. They can also compare segments and assemble supporting context without waiting for a fresh manual analysis request.
But the alert is not yet an opportunity, and correlation is not a causal explanation. A broken event, a changed identity rule, a traffic-mix shift, or a simultaneous release can resemble a change in customer behavior. Make instrumentation confidence and alternative explanations mandatory fields in the evidence packet. If either is weak, the next action is to improve the evidence, not to generate a feature.
The product trio still owns the consequential judgment: whether the problem is worth solving, what tradeoff is acceptable, which customer evidence is missing, and whether the likely value justifies the delivery cost. AI should remove investigative toil and expose overlooked evidence. It should not hide a strategic choice inside an automated recommendation.
Use AI to expand hypotheses without expanding waste
Generative design changes the economics of exploration. Once a measurable opportunity is clear, high-fidelity flows can be produced in hours instead of stretching across weeks. That makes it practical to inspect several possible mechanisms before production code is written.
Cheap variation also creates a new failure mode. If every stakeholder can request another screen, the team spends its saved production time reviewing undifferentiated options. The prompt should therefore ask for distinct value hypotheses, not a gallery of cosmetic alternatives.
Build the prototype brief from the evidence packet. It should contain:
Target user and context: the affected cohort, the job it is trying to complete, and the point at which friction appears.
Observed evidence: the behavioral signal, qualitative context if available, instrumentation caveats, and alternative explanations that remain open.
Value hypothesis: why a proposed mechanism should change the target behavior, stated in a form that can be rejected.
Meaningfully different mechanisms: alternatives that change how value is delivered, explained, sequenced, or experienced rather than merely changing visual treatment.
Outcome and guardrails: the primary behavior to influence and the accessibility, privacy, brand, reliability, and business constraints that every variation must respect.
Instrumentation needs: the events and properties required to tell whether people encounter, understand, use, and benefit from the proposed experience.
A useful review question is: If these alternatives perform differently, will you learn something about customer value? If the answer is no, the variations probably differ in presentation but not in hypothesis. Asking AI for more of them will not improve the decision.
Match the validation method to the uncertainty:
Concept validation addresses whether the intended user understands the proposition and considers it relevant.
Usability validation addresses whether the user can recognize the next step, complete the flow, and recover from confusion.
Production experimentation addresses whether exposure changes actual behavior under real product conditions.
Cohort-level follow-through addresses whether an immediate movement is accompanied by the activation, retention, or expansion outcome the team ultimately cares about.
Do not ask a prototype to answer a production question. A polished interaction can expose comprehension and usability problems, but it cannot establish that the experience will improve retention. Conversely, do not consume production capacity to answer a basic usability question that a prototype could resolve before engineering begins.
Define the decision rule before each validation step. State what evidence would cause the trio to advance the hypothesis, revise it, or stop. This prevents a compelling AI-generated design from becoming the default simply because it exists. High fidelity is a communication advantage, not proof of value.
Carry the discovery contract into production
The discovery-to-delivery handoff often introduces more error than the tools remove. A metric is renamed, a cohort becomes broader, a design constraint disappears from the ticket, or an experiment is configured to answer a slightly different question. The team ships quickly and then debates what the result means.
Before implementation begins, the trio should be able to point to a compact delivery contract containing:
The customer problem, target cohort, and value hypothesis.
The primary outcome metric and the metrics that protect against unacceptable side effects.
The exact event, property, identity, and segment definitions needed for evaluation.
The minimum detectable effect, meaning the smallest change that would be consequential enough to alter the product decision.
The planned exposure controls, eligibility rules, rollback conditions, and owner.
The accessibility, privacy, data-governance, reliability, and brand constraints inherited from discovery.
The result that would lead to shipping, iteration, further investigation, or rollback.
Set the minimum detectable effect before examining experiment results. The question is not merely whether a statistical difference can be found. It is whether the experiment can detect an effect large enough to matter to the decision. If realistic exposure cannot provide decision-worthy power, acknowledge that limitation. Consider a more substantial intervention, a longer evidence path, or a different validation method instead of asking an underpowered test for certainty it cannot provide.
Risky changes should be gated behind feature flags and delivered through a controlled CI/CD path. A flag limits exposure and creates a rollback mechanism; it does not, by itself, make a release an experiment. You still need stable assignment, defined eligibility, trustworthy instrumentation, and a predeclared interpretation plan.
Not every change is suitable for an A/B test. Some changes are required, too interconnected for clean isolation, or exposed to too little eligible traffic for a decision-worthy test. The discipline still applies: state the expected behavioral change, release progressively when possible, validate the instrumentation, inspect guardrails, and choose the review point before launch.
When production data arrives, evaluate more than the aggregate primary metric. Confirm that exposure and events behaved as intended. Inspect the cohorts named in the original opportunity. Check whether the result varies across important segments. Then follow the downstream activation or retention signal that justified the work. Production conditions include latency, reliability, real data, competing tasks, and repeated use; prototype enthusiasm does not remove any of them.
Finally, record the product decision and feed it back into the system. The agent should know which hypothesis was accepted, what actually shipped, what the experiment showed, and why the team chose to scale, revise, or stop. Without that context, the next automated investigation starts from activity rather than accumulated learning.
Give agents decision rights, guardrails, and a balanced scorecard
Agentic workflows become risky when a team discusses autonomy as a general capability. Decision rights need to be assigned to a defined action in a defined context. The same agent may safely monitor an established metric, recommend an investigation, prepare a prototype, and still require explicit approval before changing a customer experience.
Use the following as a starting policy, then tighten it to your data sensitivity, product risk, and operational controls:
Work
Agent role
Human decision gate
Required control
Monitor an established metric
Run continuously within approved read access
Metric definitions and alert conditions approved in advance
Access boundaries, instrumentation-health checks, and an audit log
Investigate an anomaly
Assemble evidence and recommend hypotheses
Product trio decides whether the signal represents a meaningful opportunity
Cohort context, alternative explanations, confidence, and traceable queries
Generate a prototype or implementation draft
Prepare alternatives and supporting artifacts
Design and engineering approve customer experience and technical choices
Accessibility, privacy, brand, architecture, and data-use constraints
Launch a customer-facing experiment
Prepare configuration; execute only when policy explicitly permits it
Named owner approves exposure, success criteria, and rollback path
Feature flag, eligibility rules, MDE, guardrails, monitoring, and rollback
Trigger a CRM or in-app workflow
Act only inside preapproved conditions
Owner approves audience, message, frequency, and stop rules
Consent-aligned data, bounded actions, suppression logic, and reviewable history
The key distinction is not human versus autonomous work. It is whether the action is bounded, observable, reversible, and aligned to an approved outcome. An agent can be highly autonomous inside a narrow monitoring job and strictly advisory when a decision affects customers, commitments, or sensitive data.
Three governance questions should appear in every agent brief: What may the agent observe? What may it decide? What may it change? Add the owner who reviews its reasoning, the evidence it must preserve, and the mechanism that stops or reverses an action. This turns broad principles such as decision rights, reasoning transparency, and outcome alignment into enforceable operating rules.
Measure flow, quality, outcomes, and risk together
A scorecard focused only on speed will reward premature action. A scorecard focused only on business outcomes will hide whether the operating system is actually improving. Track four dimensions:
Flow: time to insight, time to action, manual analysis effort, and waiting time between investigation, decision, validation, and release.
Decision quality: whether investigations include instrumentation checks and alternative explanations, and whether experiments have a hypothesis, MDE, guardrails, and interpretation rule before launch.
Customer and business outcomes: the relevant movement in activation, retention, expansion, or another outcome named in the contract, including differences across the target cohorts.
Risk: actions outside approved permissions, privacy or access violations, misleading analyses caused by instrumentation problems, customer-impacting errors, and rollbacks.
The relationships between these measures are diagnostic. Shorter time to insight with unchanged time to action means the decision queue is now the bottleneck. More agent-initiated initiatives with flat activation or retention means the organization has increased automation, not product value. Lower manual analysis effort paired with weaker evidence packets means the work became cheaper by discarding necessary scrutiny.
The percentage of initiatives initiated by agents can be useful as an adoption indicator, but it is a poor destination metric. The meaningful result is a shorter, more reliable path to customer and business impact. Keep outcome measures beside time-to-insight, time-to-action, agent-initiated work, and manual analysis effort so local efficiency cannot masquerade as progress.
Start with one bounded learning loop
Do not begin by making every product workflow agentic. Choose one recurring, measurable problem in a trusted part of the data, such as onboarding friction, activation, or retention for a defined cohort. Then roll out the operating model in sequence:
Timestamp the current stages from signal detection through decision, validation, release, and post-launch review. This establishes where work actually waits.
Stabilize the outcome, cohort, event, and segment definitions. If the instrumentation is not trustworthy, repair it before automating interpretation.
Run the agent in read-only, recommendation mode. Require the standard evidence packet and audit whether its conclusions can be reproduced.
Connect approved investigations to the prototype brief. Ask the product trio to select among distinct hypotheses and document why.
Carry the selected hypothesis into the delivery contract, feature flag, instrumentation plan, and evaluation rule.
Permit automated actions only after the team has defined bounded permissions, monitoring, stop conditions, ownership, and rollback.
Review whether the loop became faster without weakening decision quality, customer outcomes, or governance. Expand the model only where that balance holds.
If an agent cannot show how it reached a conclusion, keep it in an investigative support role. If the team cannot state what result would change its decision, pause the experiment design. If cycle time falls but no relevant outcome improves, revisit the opportunity selection and hypothesis quality rather than adding more automation.
For your next active product problem, write the outcome contract before requesting an AI analysis or prototype. Give the agent a bounded investigation brief, require the trio to compare meaningfully different hypotheses, and move the chosen hypothesis into production without changing its metric or cohort. That single end-to-end loop will tell you more about your AI readiness than a long inventory of tools.
The test is straightforward: if AI helps you reach a consequential, auditable product decision sooner and learn from the result, it has accelerated product development. If it merely creates more things to review, it has accelerated output.
Build vs. buy is a decision that never truly goes away, and with AI reshaping the economics of software, I’m revisiting this question more frequently—and with more nuance—than ever. The temptation to “just build it” is real when prototypes are cheaper, shipping feels faster, and small tools can rival big platforms. But the real decision has never been about code; it’s about value, data, and long-term responsibility.
Across product orgs at every stage, I see the same pattern: AI makes building feel easier—but it doesn’t eliminate the tradeoffs. The hard part is separating what differentiates your product from what simply supports it. That’s why I start by asking whether the capability is truly core to my value stream, and then I force myself to reason about ownership and maintenance, not just velocity.
My rule of thumb remains simple: If something isn’t core to your value stream, don’t build it. And even when it is core, vendors may still be better positioned—especially for payments, invoicing, and infrastructure. Those domains carry deep operational complexity, continuous compliance, and reliability requirements that are easy to underestimate and painful to own.
Here’s how this plays out for me. I would never build my own blogging platform. I moved from WordPress to Ghost, because publishing isn’t where I differentiate, and the long tail of upgrades, security, and performance is a drag on focus. The platform does the job, my audience gets a better experience, and my team avoids owning commodity maintenance work.
On the other hand, I did build my own task management system—despite the abundance of excellent tools like Trello, Evernote, and OmniFocus. For me, tasks, notes, and workflows are deeply personal and idiosyncratic. I wanted my system to reflect how I think, plan, and communicate, with tight integration to my daily product rituals. In this case, the underlying data became the real product—and owning and controlling that data changed the equation.
That’s the heart of the decision: When the underlying data becomes the real product, ownership matters. Task management, notes, and workflows evolve into a personalized operating system. The moment your data model represents your unique value—and your future differentiation—build vs. buy is no longer a tooling choice; it’s a strategy choice.
AI is pushing this even further. Cheaper prototyping and “vibe coding” lower the cost of building. Tools like Claude Code and platforms from OpenAI make it viable to ship smaller, targeted tools that would have been uneconomical a few years ago. That expands the frontier of what teams can build without committing to a monolithic platform—and it puts pressure on vendors to improve data portability.
Which brings me to vendor lock-in. Exports aren’t always enough. When I evaluate CRMs or course platforms, I look for more than CSV dumps. I want robust, well-documented APIs, webhook coverage, import/export parity, schema transparency, and a clear migration path. I’ve seen teams drown in brittle integrations with Salesforce or HubSpot, struggle to unwind course data from Teachable, or get stuck in signature workflows around DocuSign without a clean escape hatch. Portability is table stakes now.
I treat build vs. buy as a discovery problem. Options are assumptions to test. On the build side, I run feasibility spikes: proof-of-concept integrations, latency checks, cost-to-serve models, and a sober read on maintenance. On the buy side, I trial vendors, not their marketing. I replicate a real workflow, test the edges, validate data portability, and simulate failure modes like vendor downtime or schema changes.
A word of caution on complexity: “we can build anything” is not the same as “we should build this.” Long-lived products accumulate hidden complexity over time—security, privacy, performance, observability, SRE runbooks, QA automation, documentation, and compliance. Be honest about engineering capabilities and maintenance costs, especially when uptime and regulatory exposure are in play.
My practical checklist looks like this: Is this core to our differentiation? Do we need to own the data model? How strong is data portability (APIs, webhooks, mapping, re-import)? What’s the true total cost of ownership over three years (people, ops, security, compliance)? Are there regulatory or reliability constraints better handled by a vendor? What’s the opportunity cost of not building something more strategic? And if we buy, what’s our exit plan?
Ultimately, build vs. buy isn’t just about speed or cost—it’s about core value, data ownership, and long-term responsibility. AI lowers the barrier to building, but it doesn’t erase complexity. Treat build vs. buy decisions like any other discovery effort: test assumptions, prototype, and validate before committing. Ask not just can we build it, but should we own it?
If you’re wrestling with vendor lock-in, fielding pressure to “just build it,” or rethinking your stack in an AI-first world, this lens will help you ask better questions before you commit. And if you’re exploring targeted builds alongside platforms like Stripe, Dropbox, Obsidian, or Ghost, I’d love to hear what’s working for you and where portability remains a hurdle.
Trust is the currency of any high-stakes AI product, and nowhere is that more true than in healthcare. I recently dug into how Healio built an AI assistant for physicians—an audience that can’t afford to be wrong—and it’s a masterclass in balancing accuracy, transparency, and speed without compromising credibility.
Healio, a 125-year-old medical publishing company, set out to create Healio AI to help clinicians prepare for patient care. From the outset, their guiding principle was simple: physicians won’t trust you until you prove it. That lens shaped every decision—from discovery and prototyping to architecture, evaluation, and ongoing validation.
Discovery started with a survey of 300 healthcare professionals to understand real-world needs at the point of care. The headline insight: physicians primarily want AI for preparation, not bedside use. Even more surprising, the top ask wasn’t purely diagnostic support; it was help with patient communication and empathy—translating complex information into clear, accessible conversation.
Momentum mattered. After beginning with Figma mockups to validate workflows, the team built a working prototype in a single weekend using Cursor. That velocity wasn’t about cutting corners; it was about proving value quickly, reducing ambiguity, and iterating with concrete feedback from physicians.
Under the hood, the system employs RAG and hybrid search—combining lexical search, vector search, and semantic search across multiple trusted sources like PubMed. As any PM who has integrated biomedical literature knows, "just use PubMed" isn’t simple—there are five different ways to access the same data, each with trade-offs. The team made pragmatic choices to balance freshness, coverage, latency, and cost while preserving trust in source quality.
Designing for trust extended all the way to the citation UX. The team leaned into citations that physicians actually trust: subscripts, hover states, and progressive disclosure. This gave clinicians verifiable threads back to source material without overwhelming the core interaction, aligning with how experts want to audit evidence under time pressure.
Evaluation wasn’t left to chance. They stood up eight LLM judges for evals: safety, medical accuracy, faithfulness, relevancy, completeness, reasoning, clarity, and overall quality. Just as importantly, they treated those signals as directional, not definitive. In a high-stakes domain, physician feedback trumps LLM-as-judge feedback—so they complemented automated evals with direct reviews from practicing clinicians to calibrate quality and reduce hallucinations.
On the safety front, the team implemented HIPAA compliance and input guardrails for masking personal health information. That choice reflects strong data governance and privacy-by-design thinking: protect PHI by default, constrain prompts to safe boundaries, and make compliance a first-class citizen in the product architecture.
They also addressed monetization without compromising experience. Serving contextual ads while the LLM processes queries is a practical approach that preserves physician workflow efficiency and creates a clear, non-intrusive revenue model.
Critically, the work didn’t stop at launch. The Healio Innovation Partners provide ongoing discovery and validation, ensuring the system evolves with physician needs and the medical evidence base. This is the operating cadence you want for any AI product that sits at the intersection of safety, accuracy, and fast-changing knowledge.
My takeaways for building AI in high-stakes domains: prioritize retrieval-first pipelines over model cleverness; couple RAG with hybrid search across vetted sources; design citations that earn trust at a glance; use eval-driven development, but let domain-expert feedback be the ultimate judge; and embed regulatory compliance into your product strategy from day one. If trust is your North Star, this is a playbook worth emulating.
I’ve led product teams through countless discovery cycles, and nothing has accelerated our learning loops like AI. By weaving AI into our continuous discovery practice at HighLevel, I cut time-to-insight, reduce risk earlier, and keep our product strategy relentlessly focused on customer outcomes.
AI streamlines product discovery by accelerating research, prototyping, and validation, enabling teams to make faster, smarter, and user-driven decisions.
In the research phase, I use gen ai and LLMs for product managers to synthesize interviews, cluster themes, and surface unmet needs in minutes instead of days. Pairing those qualitative insights with behavioral signals in Amplitude analytics helps me spot high-intent cohorts and friction points at scale, so our problem framing is both human-centered and data-backed.
From there, I translate insights into crisp hypotheses and prioritize with the Kano Model and outcomes vs output OKRs. To keep experiments honest, I define a minimum detectable effect (MDE) up front and design A/B testing plans that reflect realistic traffic and seasonality, ensuring our decisions are statistically grounded rather than anecdotal.
Prototyping is where gen ai for product prototyping really shines. I spin up multiple UX flows, UI copy variants, and edge-case scenarios using prompt engineering, then iterate with rapid feedback from product trios. When needed, I mock in-app guides and product tours to validate onboarding concepts before we commit to code, preserving velocity without sacrificing quality.
For validation, I lean on a mix of lightweight experiments—fake-door tests, concierge pilots, and targeted A/B testing—augmented by in-product surveys via Pendo or Intercom. For AI-powered features, I apply eval-driven development to measure relevance, latency, and safety, so we can ship responsibly while maintaining the pace of learning.
This approach only works when the team is structured to move fast. Empowered product teams and product trios own discovery end-to-end, with clear guardrails around data governance, privacy-by-design, and AI risk management. That alignment lets us shift from opinions to evidence, and from output to outcomes, without friction.
If you’re getting started, pick one discovery loop to transform: automate research synthesis, prototype two to three variants with AI, and validate with a tightly scoped experiment. Instrument your analytics, track time-to-insight and time-to-prototype, and iterate your product roadmapping and sprint planning with what you learn. The payoff is immediate: faster cycles, stronger conviction, and a more user-driven path to product-led growth.
In product design, AI has shifted from novelty to non-negotiable. I’ve watched teams accelerate discovery, compress prototyping cycles, and turn ambiguous ideas into validated experiences faster than ever—without sacrificing quality or customer trust.
AI in product design has quickly moved from new to necessary. Here are the AI product design tools and approaches you need to stay relevant in this decade.
From my vantage point leading product teams, “necessary” means AI is woven throughout the product lifecycle—discovery, prioritization, prototyping, validation, and iteration—not bolted on. The goal isn’t to chase hype; it’s to build durable advantage with clear AI Strategy, disciplined execution, and measurable outcomes.
First, anchor the work in strategy. Tie every AI initiative to a specific customer problem and value proposition, then express that linkage with outcomes vs output OKRs. This keeps teams focused on real impact and avoids feature-chasing. It also sharpens product positioning and clarifies where AI can deliver competitive differentiation versus simple points of parity.
Second, upgrade discovery. I rely on AI workflows to synthesize interviews, cluster themes, and surface insights at scale. A retrieval-first pipeline—grounding models in our own data—improves factuality and reduces hallucinations. Combine this with strong data governance and privacy-by-design so insights are trustworthy and compliant from day one.
Third, make quality measurable. Adopt eval-driven development: define evaluation sets and acceptance thresholds that reflect real user tasks before you ship. Pair that with A/B testing and minimum detectable effect (MDE) discipline, so you learn quickly and confidently. Add safety guardrails (red-teaming prompts, content filters, and bias checks) to manage AI risk without slowing the pace.
Fourth, enable empowered product teams. Product trios (PM, design, engineering) should co-create prompts, prototypes, and evaluation criteria. Give designers and PMs practical tools—LLMs for product managers, structured prompt templates, and reusable components—so AI-augmented work becomes the default, not a special project.
Where does AI shine in product design today? Concept exploration and market scans, turning fuzzy opportunity spaces into crisp problem statements. Rapid wireframes and interaction ideas, using gen ai for product prototyping to explore multiple design directions in minutes. UX writing that adapts tone and reduces friction across onboarding, tooltip design, and microcopy.
It also excels at guided experiences. I’ve seen strong lifts in user activation when we pair in-app guides and product tours with context-aware suggestions. For support and education use cases, a retrieval-grounded assistant can deflect tickets, shorten time-to-value, and reinforce the product’s value proposition at the exact moment a user needs help.
Voice is another frontier. A well-scoped voice AI agent can accelerate complex workflows (think data entry or multi-step configurations) when hands-free is faster or more intuitive. Just be intentional about when agentic AI adds net value versus when a simple UI tweak would do.
On the tooling side, my AI product toolbox is pragmatic and modular. For analytics and learning loops, Amplitude analytics and Pendo help quantify behavior changes and retention analysis. For in-product engagement and feedback routing, Intercom and HubSpot integrate cleanly with LLM-driven tagging and summarization. For ideation and automation, I use a ChatGPT connector and Claude Code for quick scripts, data wrangling, and prompt experiments. The constant: a retrieval-first pipeline that grounds models in approved knowledge and maintains context window management at scale.
Risk management is built in, not bolted on. Set clear AI risk management policies, catalog model and data dependencies, and document decisions. Align with regulatory compliance requirements early, and keep an audit trail of prompts, datasets, and eval results. That’s how you move fast without breaking trust.
If you’re getting started, begin small: pick one high-friction workflow, add a retrieval-grounded copilot, and measure the lift. Use the results to inform product roadmapping and sprint planning, then scale to adjacent use cases. With disciplined discovery, sharp evaluation, and the right tooling, AI becomes a force multiplier for product teams and a clear win for customers.
I love real-world AI that ships, scales, and actually solves painful customer problems. This story checks every box. As a product leader who has brought agentic AI to production environments, I was captivated by how a small, focused team at Perk took a no-code voice AI prototype and turned it into a system that reliably makes 10,000+ calls per week to prevent failed hotel payments.
What happens when you combine a real customer problem, a no-code prototype, and a team willing to listen to every single call?
Steven Payne (Product Manager), Gabriel Stock (Senior Engineering Manager), and Philipe Steiff (Senior Software Engineer) from Perk share how they built a voice AI agent that calls hotels to verify virtual credit card payments, preventing travelers from arriving to find their rooms unpaid. This is a textbook example of linking operational pain to a high-leverage AI solution.
What started as a hackathon experiment in Make.com became a production system handling over 10,000 calls per week across multiple languages. Along the way, the team learned hard lessons about prompt engineering for voice (numbers, pronunciation, and a very "Karen-like" first version), how to break a single monolithic prompt into structured conversation stages, and why listening to actual calls beats any amount of theorizing.
From a product management perspective, this approach aligns perfectly with eval-driven development and continuous discovery. Structure the problem, instrument aggressively, ship safely, then listen—deeply—to real interactions. In my own teams, I’ve seen that nothing accelerates iteration on agentic AI like closing the loop between qualitative call reviews and quantitative evals.
They built a working prototype without writing a single line of backend code.
They structured the call into discrete stages (IVR, booking confirmation, payment) to improve reliability.
They created two eval systems: one for call success classification, another for conversational behavior.
They scaled from five calls a day to tens of thousands per week while maintaining quality.
This is a detailed look at building AI for real-time human interaction—where the stakes are high and the feedback is immediate.
Guests: Steven Payne, Product Manager, Perk; Gabriel Stock, Senior Engineering Manager, Perk; Philipe Steiff, Senior Software Engineer, Perk.
What stood out to me was how Perk's team identified an AI use case by connecting prior experimentation with a real operational problem. Why they chose Make.com for prototyping—and shipped to production without touching backend code—underscores how far no-code can take you when paired with crisp problem framing. The evolution from a single prompt to structured conversation stages (IVR handling, booking confirmation, payment request) is exactly how you harden agent behavior for production.
Breaking up the agent's task dramatically improved reliability. They also built two eval systems: classification for success rates and LLM-as-judge for conversational behavior. Even with automation, the team still listens to calls manually—a practice I strongly endorse for uncovering edge cases, trust issues, and UX nuances that dashboards can’t show.
The challenge of prompt engineering for voice—numbers, booking references, and text-to-speech markup—was non-trivial. Expanding to German revealed that prompts in native language improve results. And, as often happens with operations-heavy rollouts, this project uncovered other operational problems they didn't know existed—valuable signal for the roadmap.
Resources & Links: Perk. Make.com — No-code automation platform used for the prototype. Twilio — Voice/telephony provider. Eleven Labs — Text-to-speech provider (used in early experiments).
Chapters: 00:00 Introduction to the Team; 01:54 Understanding PERK's Mission; 02:59 Challenges in Travel Booking; 07:27 AI Solutions for Customer Care; 09:52 Prototyping with AI and Voice; 17:00 Implementing AI in Production; 25:51 Learning Through Trial and Error; 26:40 Prompting Challenges and Solutions; 27:58 Iterating on Prompts and Evaluations; 30:08 Scaling and Production Challenges; 32:43 Advanced Evaluation Techniques; 35:32 Real-World Applications and Success; 49:07 Future Directions and Expansion; 53:53 Conclusion and Team Reflections.
My product takeaways: Start with clear operational pain and measurable outcomes (e.g., payment verification). Use no-code to validate quickly, then progressively harden. Treat voice AI like any production system: break it into deterministic stages, add guardrails, and measure both outcome and behavior. Pair automated evals with hands-on reviews. And when going multilingual, write prompts in the native language—your accuracy will thank you.
If you’re exploring agentic AI for operations, this is the blueprint: tight scoping, Make.com for speed, Twilio for reliability, structured prompts for control, and an eval-driven loop to scale quality with confidence.
I’ve spent much of my career compressing the distance between a napkin sketch and something real customers can touch. At HighLevel, my product teams use generative AI to validate ideas faster, reduce risk earlier, and win stakeholder trust with evidence instead of slides. The goal isn’t to be flashy—it’s to be precise, testable, and repeatable.
Today, you can build it before you pitch it. AI prototyping can turn ideas into clickable demos in hours. Here are some tools to try and steps to follow.
I start every AI prototyping sprint by sharpening the problem statement and the outcome we care about. That means being explicit about the target user, jobs-to-be-done, and the riskiest assumptions. I define a minimum detectable effect (MDE) and tie it to outcomes vs output OKRs so everyone aligns on what “good” looks like before we touch a tool.
From there, I move from sketch to interface. I capture a rough flow (whiteboard, tablet, or even paper) and generate UI variations with my AI product toolbox—tools that translate structure into components and screens. I’ll iterate on information hierarchy and copy until the narrative supports the core job, borrowing techniques from UX writing. For product managers leaning into LLMs for product managers, this phase is about speed to feedback, not perfection.
Next, I wire data and logic. I connect a lightweight backend or spreadsheet, stitch in a CRM integration if needed, and add LLM calls through a ChatGPT connector or Claude Code. If the concept benefits from multi-step autonomy, I introduce agentic AI to orchestrate tasks across APIs. CustomGPT workflows help me encapsulate business rules so the demo behaves consistently in user paths we care about.
Governance is not optional at this stage. I apply privacy-by-design defaults, document data governance decisions, and run a quick AI risk management pass: input validation, prompt safety, rate limits, and fallback responses. This keeps the prototype credible and prevents false positives from polluting stakeholder perception.
With a click-through in hand, I instrument the experience so learning compounds. I drop in Amplitude analytics to track activation, task completion, and drop-off, and set up simple A/B testing when there’s a meaningful design or copy choice. This makes the prototype a learning vehicle, not just a demo.
Then I get it in front of users—fast. Five targeted conversations will beat fifty internal opinions. I run structured product discovery interviews, observe time-to-value, and capture objections. This is where empowered product teams shine: we make changes in real time, re-run the flow, and document what moves the needle for product-led growth.
When speed matters, I use a four-hour cadence: Hour 1 for problem framing and MDE; Hour 2 for sketch-to-UI generation; Hour 3 for data wiring and AI logic; Hour 4 for instrumentation and user walkthroughs. By the end, we have a clickable demo, preliminary analytics, and a clear decision on whether to advance, pivot, or park.
Finally, I translate insights into a concise artifact: the hypothesis we tested, the signal we observed, the trade-offs we made, and the next sprint plan for product roadmapping and sprint planning. The point is not to be right on the first try; it’s to learn precisely, cheaply, and quickly enough to invest with conviction.
If you adopt this approach, you’ll find that stakeholder management becomes easier, team energy rises, and your roadmap earns credibility. Build it before you pitch it, and let real interactions—not wishful thinking—do the heavy lifting.
