Tag: AI Strategy

How to Prove the ROI of an AI Product Before You Scale It

Your AI product is getting used. The demos land well, task completion is improving, and internal enthusiasm is high. Then the CFO asks a harder question: what changed in the business because this product exists?

You cannot answer that question with prompt volume, response quality, adoption, or tickets touched. You need a measurement system that separates activity from incremental value, counts the full operating cost, and makes risk visible before a rollout gets larger. Here is how to build one.

Start with the decision your ROI model must support

ROI is not a retrospective slide assembled after launch. It is a decision rule. Before development begins, decide what evidence would justify launching, scaling, redesigning, rolling back, or retiring the capability.

That distinction changes the conversation. Instead of asking whether the agent is accurate enough or popular enough, you ask whether a measurable change in customer behavior produces a measurable business result without crossing an unacceptable risk threshold.

Build a driver tree with four levels:

Company outcome: revenue growth, lower cost to serve, or reduced business risk.
Customer outcome: the user completes a valuable job, reaches value sooner, or resolves a problem without unnecessary effort.
Product behavior: the AI capability changes conversion, expansion, self-service completion, containment, handle time, or escalation.
Controllable lever: the team changes the workflow, model behavior, conversation design, human review, or product guidance.

The chain matters because a model metric is rarely a business metric. Better answer quality may improve task completion, which may improve trial-to-paid conversion. The ROI case depends on the full chain, not the first link.

Value path	Business outcome	Leading evidence	Guardrails
Revenue	Higher conversion, average order value, or expansion	Time-to-first-value and self-service completion	Errors, complaints, and policy violations
Cost	Lower cost to serve	Containment, deflection, and reduced handle time	Escalations, false resolution, and downstream customer harm
Risk	Lower frequency or impact of harmful failures	Human-review events and detected violations	False positives, false negatives, hallucinations, and security breaches

Choose one primary value path for the investment case. Revenue, cost, and risk can all appear on the scorecard, but declaring all three as primary makes it too easy to rescue a weak result with whichever metric moved after launch.

A support agent, for example, may appear successful because it contains more conversations. But containment is only valuable if customers actually resolve their problems. A conversation that never reaches a human can reduce measured support volume while increasing complaints or churn risk. This is why revenue, cost, and risk measures must be evaluated together.

Write the measurement contract before you build the dashboard

A measurement contract is a short agreement among product, data, finance, and the operational team affected by the AI workflow. It prevents the definitions, cost boundaries, and success thresholds from changing after results arrive.

Your contract should answer these questions:

Who is eligible? Define the users, accounts, tasks, channels, and exclusions. Do not mix workflows with materially different economics.
What is the intervention? Name the AI capability and the version being evaluated. A model, prompt, retrieval pipeline, policy, or escalation change can alter the result.
What is the primary outcome? Select the business metric that determines whether the hypothesis passed.
What are the leading indicators? Use measures such as time-to-first-value, containment, and self-service completion to diagnose movement before lagging results mature.
What are the guardrails? Predefine acceptable limits for errors, hallucinations, false positives, false negatives, escalations, complaints, security events, and policy violations.
What is the baseline? Freeze the comparison period or control group before exposing the eligible population to the capability.
How will incrementality be proven? Specify the experiment, holdout, assignment unit, and minimum detectable effect.
What costs count? Agree on model or API consumption, labeling, evaluation, human review, and ongoing oversight before calculating value.
What action follows each result? Record the thresholds for launch, scale, redesign, rollback, and retirement.

The contract should distinguish an outcome OKR from an output OKR. Shipping the agent, generating responses, and increasing feature use are outputs. Improving conversion, lowering verified cost to serve, or reducing harmful failures are outcomes. Outputs can explain what happened, but they cannot establish value on their own.

Instrument the complete journey, not just the conversation

An AI log tells you what the model did. An ROI dataset must also tell you what the user did next.

Connect the journey from eligibility to business outcome:

The user or account became eligible for the capability.
The AI experience was offered, viewed, and engaged.
A task was attempted, completed, abandoned, or repeated.
A response was accepted, corrected, regenerated, or sent for human review.
The interaction was contained, escalated, or handed to another workflow.
The downstream conversion, expansion, support, retention, or complaint event occurred.
The associated model cost, labeling work, and human-oversight cost were recorded.

Carry a stable user or account identifier, experiment assignment, agent version, and journey identifier across those events. Without that connective tissue, the team may have an impressive agent dashboard and no defensible way to attribute a business outcome to the experience.

Use behavioral analytics and session replay to understand why a metric moved. Use journey mapping and retention analysis to locate the friction worth solving in the first place. Product tours and in-app guidance can then help eligible users reach a validated workflow. This creates a closed loop from journey friction to experiment and measurable outcome, instead of a collection of disconnected AI metrics.

Calculate economic value without turning activity into savings

Start with net business value:

Net business value = incremental revenue + cost avoided – total operating cost – quantified risk loss

If finance requires an ROI percentage, divide net business value by the agreed investment base. Keep both the numerator and denominator visible. A percentage without its cost boundary is easy to inflate and hard to audit.

Count only incremental revenue

Do not credit the AI product with every transaction it touched. Credit it with the difference between the exposed population and the valid control or holdout.

A practical revenue calculation is:

Incremental revenue = eligible volume x measured outcome lift x value per additional outcome

The measured outcome might be trial-to-paid conversion, self-service upsell, average order value, or expansion. Use the same eligibility definition, attribution window, and revenue treatment for the intervention and control. If the AI experience merely appears somewhere in a successful journey, that is influenced revenue, not proof of incremental revenue.

Separate capacity from cashable savings

Cost claims require more care than a deflection count. A contained interaction may create capacity without reducing expenditure. That capacity can still be valuable, but it should not be presented as cash savings unless spending actually changes.

Capacity created: employees have time available for other work, but the existing cost base remains.
Variable cost avoided: the company no longer incurs a cost that would have grown with each additional interaction.
Cashable savings: an approved budget, vendor charge, or staffing requirement is actually reduced.

Report these separately. Otherwise, the same saved minute can be counted once as employee capacity and again as reduced spend.

Validate that a deflected task was resolved, not abandoned or displaced to another channel. Then calculate avoided cost from the incremental lift in verified resolution, not the total number of conversations the agent handled.

Include the operating costs that make the agent dependable

Model or API cost is only one part of the investment. Include labeling, evaluation, human review, and operational oversight. If a safer workflow requires more review, that review is part of the product’s economics, not an external inconvenience to exclude from the model.

Segment cost by agent, workflow, and outcome. Cost per response is useful for infrastructure management, but cost per verified successful outcome is the better economic unit. A cheap response that triggers retries, escalations, or corrections may be more expensive than a higher-cost response that completes the job.

Do not bury risk inside an average ROI number

Risk adjustment should make uncertainty visible, not create false precision. Use three layers:

Hard guardrails: security and policy conditions that trigger containment or rollback regardless of financial upside.
Observed risk indicators: error, hallucination, escalation, complaint, false-positive, and false-negative rates tracked by workflow and cohort.
Financial adjustment: expected loss deducted from net value only when the probability and impact assumptions are credible enough for finance and risk owners to accept.

Do not let a low-frequency, high-consequence failure disappear inside a high average success rate. If the downside cannot be defensibly monetized, keep it as an explicit decision constraint rather than assigning it a convenient dollar value.

Prove incrementality before claiming impact

The strongest ROI calculation still fails if the attribution is weak. A before-and-after improvement may come from seasonality, pricing, traffic quality, a support policy change, or another product release. The AI capability needs a counterfactual: what would have happened to comparable eligible users without it?

Use an A/B test or holdout whenever the product and risk profile allow it. Make these choices before launch:

Assignment unit: Randomize at the level where the outcome occurs. If expansion is measured per account, account-level assignment can prevent users in the same customer organization from receiving conflicting experiences.
Primary outcome: Pick the metric that determines success and keep diagnostic metrics secondary.
Minimum detectable effect: Precompute the smallest lift worth detecting based on the baseline, available population, and business value. If the experiment cannot detect a decision-relevant change, extending the metric list will not fix it.
Guardrails: Test quality, escalation, complaints, security, and policy outcomes alongside the primary metric.
Analysis population: For a product-level ROI claim, analyze eligible users according to their assigned experience. Looking only at people who voluntarily used the agent introduces selection bias.
Measurement horizon: Keep the holdout long enough to observe the outcome named in the contract. Leading indicators can guide iteration, but they should not be substituted for retention, churn, Net Recurring Revenue, or other lagging outcomes.

If randomization is not practical, use a fixed holdout or a frozen comparison period and document the limitations. A weaker design can still inform a decision, but the ROI claim should carry less confidence. Do not quietly promote correlation to causation because the rollout has executive attention.

Interpret the result as a system. Suppose self-service completion rises but the business outcome does not. The agent may be solving a low-value task, attracting users who would have converted anyway, or shifting effort to a later step. If conversion improves while complaints or policy violations cross the guardrail, the value hypothesis may be valid but the implementation is not ready to scale.

This is eval-driven development applied to product economics: define acceptable behavior and business success, measure both under controlled conditions, diagnose the failures, and repeat the test after a meaningful change.

Turn ROI into a portfolio operating system

A one-time business case goes stale as models, prompts, traffic, user behavior, and operating costs change. Maintain an Agent Analytics view for every production capability.

Each agent scorecard should show:

The primary business outcome and current experiment result.
Leading journey metrics from eligibility through verified completion.
Revenue contribution, cost avoided, and total operating cost using the agreed definitions.
Quality and risk guardrails, including escalations and human-review events.
Performance by relevant customer, task, and journey cohort.
The agent, model, policy, and workflow version associated with the result.
The current decision status: exploring, launching, scaling, redesigning, contained, or retiring.

Use the dashboard to make portfolio decisions, not merely to report trends:

Scale when the primary outcome clears the precommitted threshold, guardrails hold, net value is positive, and the result remains credible across the cohorts that matter.
Redesign when leading indicators improve but the business outcome does not, or when human review and escalation erase the economic gain.
Contain or roll back when a hard security, policy, or customer-harm threshold is breached, even if average financial performance is positive.
Retire when controlled measurement shows no decision-relevant incrementality or when dependable operation costs more than the value created.

Review operational signals with frontline teams because they can explain patterns hidden by aggregate metrics. Review portfolio value in QBRs with product, data, finance, and risk owners so investment follows evidence rather than novelty.

Only accelerate adoption after the workflow has demonstrated unit value. In-app guides, product tours, and lifecycle nudges can bring more eligible users into a validated flow. Measure whether those interventions increase the business outcome, not merely clicks or agent sessions. Scaling exposure to an unproven workflow scales its cost and risk as readily as its potential benefit.

Key takeaways

Treat ROI as a precommitted decision rule for launch, scale, redesign, rollback, or retirement.
Connect model behavior to customer behavior and then to revenue, cost, or risk through a driver tree.
Freeze the baseline, cost boundary, guardrails, attribution method, and success thresholds before results arrive.
Credit only incremental revenue and verified avoided cost. Keep created capacity separate from cashable savings.
Include model consumption, labeling, evaluation, human review, and oversight in the operating cost.
Use controlled experiments or holdouts, with a decision-relevant minimum detectable effect, to separate causal impact from correlation.
Keep severe risk conditions as explicit constraints when they cannot be responsibly converted into a financial estimate.
Scale adoption only after the AI workflow has shown positive unit value under acceptable risk.

Pick one high-friction customer journey and complete its measurement contract before the next roadmap review. If the team cannot name the baseline, control, primary outcome, cost boundary, guardrails, and decision thresholds, the capability is still an exploration. Label it honestly, instrument it properly, and earn the right to make an ROI claim.

References

May 15, 2026

AI-Enabled Enzymatic Recycling: A Product Leader’s Playbook

You have an AI-enabled materials proposal in front of you, a promising set of enzyme candidates, and a difficult decision: fund another round of discovery or start building toward industrial scale. The candidate sequences may be impressive, but they are not yet the product.

Your decision should turn on whether the full system can repeatedly transform a defined waste stream into usable monomers at an economically viable cost. That framing connects model performance, laboratory evidence, process engineering, and commercial reality before an exciting demonstration becomes a stranded pilot.

Define the product around recovered monomers

Only 10% of the plastic manufactured gets recycled. That ceiling is not merely a sorting or consumer-behavior problem. Traditional recycling commonly shortens polymer chains instead of restoring their original molecular building blocks, so the resulting material can lose quality and move toward downcycling.

Enzymatic recycling changes the intended output. An engineered enzyme can deconstruct a polymer into its original monomers, which can then become inputs for new, high-quality plastic. The difference is fundamental: the product is not processed waste or a smaller plastic fragment. It is recovered molecular feedstock.

This distinction gives you a better product boundary. A generated protein sequence is a feature. An enzyme that shows activity in one assay is a technical result. The product is a repeatable monomer-recovery system with a defined input, output, operating envelope, and cost structure.

Before approving a roadmap, require the team to define five contracts:

Input contract: Which polymer, packaging format, mixture, and contamination profile will the process accept? “Mixed plastic” is not a specification. Name the included materials and the variation the system must tolerate.
Transformation contract: Which polymer bonds must the enzyme break, and what conversion and selectivity must the reaction demonstrate?
Output contract: Which monomers will be recovered, what downstream use must they support, and how will the team determine that the output is suitable for that use?
Operating contract: What reaction conditions, throughput, energy consumption, and process controls must hold outside a small laboratory assay?
Economic contract: Which cost per ton must the integrated process approach, and which assumptions currently separate measured economics from projected economics?

Selectivity is especially important. An enzyme can target a particular plastic within a mixed waste stream, potentially reducing the need to treat every input as chemically identical. But selectivity does not make an undefined waste stream manageable. The process still needs to know which target material is present, whether the enzyme can reach it, and how the desired products will be recovered.

Write the product brief in one sentence: For this defined feedstock, transform this polymer into these monomers, within this operating envelope, output specification, and cost boundary. If a number is unknown, leave a visible blank and assign an experiment to fill it. Do not hide the uncertainty inside a broad ambition such as “make plastic circular.”

Build the AI as a closed learning system

AI changes the economics of searching enzyme-design space. Protein language models can generate candidates, multi-step agents can coordinate specialized tasks, and computational evaluations can eliminate weak options before scarce laboratory capacity is used. Advances in protein structure prediction have expanded what can be explored, but prediction does not remove the need for physical validation.

The useful architecture is therefore not a model that emits sequences. It is a closed loop in which every physical result makes the next design round better. Rhea’s Factory combines protein language models, an agentic pipeline, domain constraints, and proprietary wet-lab feedback. The product lesson is broader than any one implementation: generation, evaluation, experimentation, and learning need to operate as one traceable system.

Encode the objective. Convert the product contract into machine-readable constraints: target polymer, desired products, acceptable operating conditions, and the metrics that will decide whether a candidate advances.
Generate candidates. Explore multiple plausible designs rather than optimizing immediately around the first promising family.
Apply computational gates. Reject candidates that violate explicit constraints, preserve the reasons for rejection, and rank the remaining candidates for laboratory use.
Run controlled wet-lab experiments. Test candidates under recorded conditions and capture successes, failures, and inconclusive results.
Update domain predictions. Use the measured outcomes to improve ranking and candidate selection for the next round.
Feed process evidence back into discovery. When a candidate struggles under reactor or feedstock conditions, turn that failure into a new design constraint instead of treating it as a separate engineering problem.

Agentic AI is valuable here because the workflow is multi-step, not because an agent should make every decision autonomously. At each handoff, define the required input, expected output, validator, and failure behavior. A generation step should not advance an incomplete candidate. A computational score should not be presented as a laboratory observation. A promising assay should not silently become a scale claim.

Exploration also needs an explicit lane. Higher model-sampling temperatures can produce more unusual enzyme candidates and reach beyond the safest local variations. Controlled model “hallucination” can be useful during candidate exploration when downstream guardrails prevent novelty from being mistaken for evidence.

Separate the candidate portfolio into three buckets: improvements near known winners, adjacent designs that test a clear hypothesis, and high-variance exploration. Give each bucket a deliberate laboratory budget. Raise sampling temperature only in the exploratory lane, and never allow generated assay values, reaction outcomes, or scale results into the measured-data record.

The durable advantage sits in the feedback data. In a narrow, high-signal domain, even hundreds of relevant proprietary laboratory observations can support a useful domain prediction model. That is not a general claim that small datasets are always sufficient. It means contextual quality can matter more than indiscriminate volume when the problem, assay, and outcomes are tightly defined.

For every experiment, preserve enough context to make the result reusable:

The enzyme identity, sequence version, and design lineage.
The target polymer, material format, mixture, and relevant contamination profile.
The assay and protocol version used for the test.
The reaction conditions and duration.
The measured conversion, selectivity, yield, and uncertainty available from the experiment.
The full result, including failure, no-result, and inconclusive outcomes.
The relationship between the candidate, computational evaluations, physical test, and model or data release.

A spreadsheet of winning sequences is not a data moat. A traceable record of why candidates were proposed, how they were tested, what failed, and how each result changed the next decision can become one.

Use stage gates that end in physical evidence

AI product teams often gravitate toward a model leaderboard because it creates a clean sense of progress. Enzymatic recycling does not have one adequate master score. A candidate can look structurally plausible and fail in the lab. It can perform in a controlled assay and miss the required throughput. It can convert the polymer and still lose economically once the rest of the process is counted.

Use a hierarchy of evidence that moves from design compliance to laboratory performance, operating fit, and scale economics:

Gate	Decision question	Required evidence	Red flag
Design compliance	Does the candidate satisfy the stated target and pipeline constraints?	Deterministic checks, recorded constraint evaluations, and candidate provenance	A candidate advances mainly because it appears novel
Wet-lab performance	Does the enzyme convert the target with the required selectivity under defined conditions?	Repeatable measured observations, including negative and inconclusive runs	Only the best run is retained or shared
Operating fit	Does useful performance hold within the intended controlled, low-temperature process and throughput requirements?	Process measurements tied to reaction conditions, conversion, yield, throughput, and energy use	Activity is reported without the process context needed to interpret it
Scale economics	Can the integrated system move toward cost parity with inexpensive oil-based plastic?	A cost and energy model tied to measured inputs, with assumptions and sensitivities exposed	Commercial viability is inferred from enzyme activity alone

Set pass, hold, and stop conditions before seeing the result. Otherwise, an interesting candidate will repeatedly earn one more experiment while the commercial requirement drifts. Relative improvement is useful for learning, but an enzyme that is twice as good as an unusable baseline may still be unusable. Every relative metric should sit beside the absolute requirement it is meant to approach.

Keep conversion, selectivity, yield, throughput, and energy per ton separate. Combining them too early into a single score can conceal the actual tradeoff. A team should be able to show why it is advancing a faster candidate with lower selectivity, or a more selective candidate with a different operating burden, without claiming that the candidates are equivalent.

Three common metric substitutions deserve direct scrutiny:

Low reaction temperature is not automatically low total energy. Count the energy demands of the complete process rather than the enzyme reaction in isolation.
Polymer conversion is not automatically usable monomer recovery. Measure whether the desired output can be recovered to the specification required downstream.
Bench performance is not automatically scaled performance. Treat increasing process scale as a new evidence gate, not a routine deployment step.

My rule is simple: model output can earn laboratory time; only measured process evidence can earn scale capital.

Plan the roadmap backward from cost parity

The commercial benchmark is unforgiving. Enzymatic recycling ultimately has to compete with inexpensive oil-based plastic production. A greener reaction that cannot approach a viable delivered cost will remain dependent on special conditions rather than becoming a broadly adopted circular process.

Build the economic model while discovery is still underway. At minimum, separate these cost lines:

Feedstock acquisition, sorting, and rejected material.
Preparation required before the enzyme can act on the target polymer.
Enzyme production, delivery, useful lifetime, and replacement.
Reactor capacity, reaction time, process control, and energy.
Monomer recovery and purification.
Waste handling, downtime, and variability in plant utilization.

Do not wait for perfect values. Use ranges, label each input as measured or assumed, and run sensitivity analysis. The purpose is to identify which uncertain variable can kill the business case. If enzyme lifetime dominates cost, another candidate-generation run may be rational. If purification dominates, generating thousands of additional sequences may be a distraction from the real constraint.

Pair every scientific milestone with an industrial question:

Discovery gate: Is activity and selectivity reproducible enough to justify process work?
Process gate: Does the candidate perform inside the intended operating envelope rather than only under a convenient assay condition?
Feedstock gate: Does performance survive representative material formats and mixtures, including difficult packaging such as clamshells?
Demonstration gate: Can the system sustain the required material flow, output quality, and energy profile at a scale that tests the major engineering assumptions?
Commercial gate: Does the cost case remain credible when feedstock composition, utilization, throughput, and other sensitive inputs move away from the preferred case?

A planned 5,000-ton demonstration plant in California illustrates why demonstration capacity belongs on the product roadmap. A plant is not simply a larger laboratory. It tests whether biology, equipment, controls, feedstock variability, and recovery operations behave as an integrated product.

Before committing meaningful scale capital, ask six kill questions:

Which assumption has the largest effect on delivered cost per ton?
Which inputs are measured, and which still come from a design estimate?
At what physical scale was each important input measured?
What fails first when the feedstock mix changes?
If enzyme performance improves as planned, which downstream step becomes the bottleneck?
Which observed result will stop, narrow, or materially redesign the program?

Expansion into additional plastics should follow the same discipline. Enzyme selectivity creates a plausible path toward enzyme blends for mixed streams, and new plastic types and mixed-plastic blends remain important development directions. Treat each added polymer as a new product vertical with its own input contract, assays, process interactions, recovery requirements, and economics. A new enzyme is not automatically a low-cost extension of the first process.

Key takeaways for your next roadmap review

Define success as repeatable recovery of specified monomers, not the generation of novel enzyme sequences.
Run discovery as a closed loop connecting product constraints, AI generation, computational gates, wet-lab measurements, and process feedback.
Treat proprietary experimental context—including failures—as the data asset; candidate count alone is not a defensible moat.
Use separate gates for design compliance, laboratory performance, operating fit, and scale economics.
Work backward from cost parity and direct the next experiment toward the assumption that most threatens the integrated business case.

For your next review, ask the team to bring one page containing the input and output contracts, a diagram of the learning loop, the current stage-gate thresholds, the experimental data schema, and a cost sensitivity model with measured and assumed inputs clearly separated. Every roadmap item should change one of those artifacts or produce evidence for a named decision.

If the team cannot fill those fields yet, that is the immediate product work. The first defensible milestone is one traceable loop from a defined industrial problem through candidate generation, laboratory measurement, and an updated cost model. Repeat that loop with increasing realism before increasing capital exposure. That is how you determine whether programmable biology is becoming an industrial recycling product rather than remaining an impressive AI demonstration.

References

Product Talk — How AI-Designed Enzymes and Agentic AI Could Finally Make Plastic Truly Recyclable

May 14, 2026

Figma’s Executive Scaling Playbook for IPO Readiness

If your company may pursue an IPO, the tempting move is to wait until the filing window is visible, then recruit executives who have done it before. That sequence solves for credentials. It does not necessarily build the decision system the business will need.

Figma took a different path. An operator who joined when the company had about 30 people and was not yet charging for its product grew into the CFO role, while the company adopted public-company habits years before its 2025 IPO. The useful lesson for you is not simply to promote insiders. It is to develop executive judgment, operating cadence, and economic instrumentation as one connected system.

Key takeaways: the playbook on one page

Expand a leader’s decision scope before expanding the title. Look for progression from doing the work, to framing the questions, to allocating resources, to improving decisions across the company.
Install public-company behaviors before the transaction demands them. Figma began operating this way three years before its IPO, using quarterly rhythms, tighter controls, a close that could withstand scrutiny, and a coherent forward-looking narrative.
Treat product, finance, and go-to-market as joint owners of the economic model. They need the same driver tree, definitions, telemetry, and assumptions before they debate pricing or investment.
Manage AI investment as a portfolio of explicit bets. Usage, customer value, cost-to-serve, decision triggers, and risks should be visible even when the underlying economics are changing quickly.
Use leadership transitions to redraw decision rights. Replacing a departing executive without reconsidering the operating model preserves yesterday’s bottlenecks.

Scale executive judgment before you scale titles

Praveer Melwani joined Figma in 2017 as its first business operations and finance hire. The company was still around 30 people and had not begun charging for the product. He became CFO in 2022 and helped lead the company through its IPO in 2025.

The important pattern is the sequence of work. Early responsibilities included building driver trees, challenging go-to-market assumptions, and establishing the mechanics of board management. Later responsibilities moved toward defining the questions the company needed to answer, directing capital, and shaping the operating cadence. The role grew because the decisions grew.

You can use that sequence as an executive-readiness ladder. It is more informative than tenure or the seniority of a candidate’s last title.

Executive mode	Work that demonstrates readiness	Failure signal to watch
Operator	Builds the model, tests assumptions, and makes the basic process reliable.	Produces accurate work but cannot explain which decision it should change.
Question-setter	Identifies the uncertainty that matters, frames options, and defines success.	Waits for the founder or another executive to determine what deserves attention.
Allocator	Connects product evidence, financial constraints, and strategic upside to resource choices.	Treats the budget as a fixed entitlement rather than a set of revisable bets.
System leader	Improves the cadence, decision rights, narrative, and judgment of the wider team.	Remains the indispensable reviewer for every important decision.

Do not promote someone merely because they are excellent in the first row. Give them work from the next row and observe what happens. Ask a strong operator to frame an ambiguous company problem, recommend where resources should move, document the trade-offs, and run the decision through the relevant functions. You are testing whether the person can create clarity beyond the boundaries of the original role.

I use a similar first-principles test when evaluating a prospective VP, especially in a function the founder does not know deeply:

Can the candidate map how the business creates and captures value?
Can they define success metrics and show where those metrics could mislead the team?
Can they explain a meaningful trade-off in plain language?
Can they describe the team and decision system they would build, rather than only the work they would personally perform?
Can they teach the executive team something useful in 30 minutes?

Run this test on your actual business context, not a generic case interview. A candidate who asks sharper questions, exposes a hidden assumption, and improves the decision has demonstrated more than someone who recites the standard playbook from a previous employer. Prior experience still matters, but learning velocity and expanding scope deserve more weight than familiarity alone.

Make IPO readiness a company cadence, not a finance workstream

Figma began behaving like a public company three years before its IPO. That is not a universal countdown for every company. The more important point is the order of operations: the habits came before the event that would test them.

Late preparation forces teams to create controls, reconcile definitions, improve forecasting, and construct a credible narrative while the stakes are already high. Early preparation turns the same work into ordinary management. It also reveals weak ownership and unreliable data while the company still has room to correct them.

Quarterly operating rhythm: Review changes in the business drivers, the assumptions behind the forecast, the resulting resource choices, and the risks that could alter the plan. A performance presentation without a decision is reporting, not an operating review.
Close and controls: Make ownership, evidence, access, and material judgments explicit. The goal is not bureaucracy for its own sake. It is to produce numbers that leaders can use without reopening the entire chain of custody every time.
Forward-looking narrative: Connect past performance to the decisions now being made. Explain what changed, why management believes it changed, what will be done next, and what evidence would invalidate that view.
Decision record: Preserve the assumptions, alternatives, owner, and follow-up trigger behind a material choice. This prevents the company from rewriting the reasoning after the outcome is known.

This discipline can accelerate decisions because product, finance, and go-to-market stop renegotiating the basic facts in every meeting. Product brings evidence about behavior and roadmap alternatives. Finance brings the model, sensitivities, and constraints. Go-to-market brings customer context, commercial implications, and execution dependencies. The executive owner makes the cross-company choice and records what would cause it to change.

A useful quarterly decision packet should answer the following questions:

<!– wp:list {

May 14, 2026

From Vision to Execution: Building Agentic, Data‑Driven Products with Real‑World Rigor

When I consider where product development is headed, one statement captures the mandate perfectly: "Eric Carlson is a Principal AI Engineer helping to shape and build Amplitude's next generation vision of of agentic and data driven product development." That vision resonates deeply with how I lead teams—anchoring strategy in behavioral analytics while enabling agentic AI to act on insights with speed, safety, and measurable impact.

Translating that vision into execution starts with clarity of outcomes. I frame driver trees that connect customer value to leading indicators—activation, engagement depth, and retention—then instrument product telemetry with Amplitude analytics and behavioral analytics to surface the moments that matter. From there, we operationalize learning with A/B testing and feature flags, ensuring each hypothesis gets a fair, observable run and that we can safely ramp what works.

Agentic AI changes the operating model. Instead of static dashboards, we design autonomous workflows that observe signals, reason over context, and take action—grounded in a retrieval-first pipeline and governed by eval-driven development. For product managers, this demands fluency with LLMs for product managers and practical prompt engineering, plus rigorous AI Strategy around data governance, privacy-by-design, and risk scoring so agents remain trustworthy under real-world conditions.

Cross-functional cadence is everything. I partner closely with Principal AI Engineers and product trios to blend continuous discovery with execution: rapid user interviews to reveal intent, opportunity solution trees to prioritize, and outcomes vs output OKRs to align incentives. The result is a system where insights are unified, decisions are explainable, and agents improve through tight feedback loops across analytics, experimentation, and production telemetry.

If you’re building toward an agentic, data-driven future, invest in a unified analytics platform, shorten the path from signal to action, and measure learning velocity as carefully as feature delivery. With the right foundations, agentic AI becomes more than a feature—it becomes a force multiplier for product strategy, customer value, and sustainable growth.

Inspired by this post on Amplitude – Perspectives.

May 13, 2026
Beyond the Product Builder Hype: How AI, org design, and joy shape PM success

I recently spent time with the debate behind the "product builder" trend—asking whether it’s the future of product management or just another wave of tech FOMO. The conversation featuring Teresa Torres and Petra Wille is a useful prompt, but what matters most is how we translate these ideas into healthy product practices inside our own organizations.

Here’s my take: the product builder movement is neither a mandate nor a fad—it’s a tool. The right question isn’t "should product managers code?" but whether leaning into building advances outcomes for our customers and our teams. In practice, that means letting interest and skill—not pressure—set the pace.

Petra captured it perfectly: "Just because I can do it — is it something I enjoy doing? And do I have enough experience to really get into the flow?" Those two tests—joy and depth—are underrated filters. I’ve seen PMs light up when prototyping or vibe coding a thin slice, and I’ve also seen well-meaning dabbling create hidden complexity that slows everyone down later.

Org design determines whether this works. It’s not about the tools—it’s about clarity of roles, healthy interfaces between product, design, and engineering, and explicit guardrails for where experiments stop and production begins. AI has raised the stakes: "AI can make unskilled work look polished. That’s a feature and a bug — executives see the shine, engineers inherit the mess." If you’ve ever watched a glossy demo turn into weeks of refactors, you know exactly what this looks like.

To avoid that trap, I deliberately separate the three layers where AI is changing product work: personal productivity, team process, and product strategy. Treating these as different stacks keeps expectations clean: a prompt that accelerates personal workflows isn’t the same as an AI-enhanced process that reshapes delivery, and neither automatically produces durable product advantage. Don’t conflate them.

Discovery remains stubbornly human. "Why discovery still requires talking to your customers (sorry)" is more than a friendly nudge. AI can broaden our search space and sharpen analysis, but it doesn’t replace qualitative conversations or the judgment that comes from pattern recognition across real customer contexts. Continuous discovery and disciplined customer interviews are still the most reliable compasses we have.

Where does "vibe coding" fit? It’s great for roughing out concepts, de-risking slices, and communicating intent when words or static mocks won’t cut it. Tools like Claude Code make this faster than ever, and familiar stacks like Ruby on Rails lower the bar for spinning up functional prototypes. But remember the design system trap: AI can make bad decisions look good on the surface. If you don’t control for architecture, accessibility, data contracts, and handoff quality, your team pays the integration tax later.

In well-set-up orgs, the output-oriented muscle memory gets rewired. When AI frees up time, strong teams reinvest it into better problem framing, sharper opportunity solution trees, and tighter product strategy—rather than simply chasing more output. That’s a leadership challenge, not a tooling problem, and it shows up quickly in how teams make trade-offs.

Here’s how I operationalize this with empowered product teams: we articulate clear boundaries for prototypes versus shippable code, define decision rights for when PMs or designers "build," and align on review gates that protect quality without stifling speed. We also make the three AI layers explicit in roadmapping and retros, so improvements to personal workflows don’t get mistaken for strategic advantage.

My distilled guidance echoes the episode’s throughline. The product builder trend isn’t a mandate — it’s a tool. Let enjoyment and skill guide who on your team leans into it. Organizational readiness determines whether AI empowers your team or creates chaos. Don’t conflate personal efficiency, process change, and product impact—they require different responses. Discovery fundamentals haven’t changed; AI helps you go deeper, not skip the work. And the real takeaway on product builders: not everyone has to build, but everyone can if they want to.

If you want to hear the full discussion that sparked these reflections, listen on Spotify or Apple Podcasts. Then tell me: where will you apply builder energy in your team—and where will you deliberately say no?

Resources & Links: Follow Teresa Torres: https://ProductTalk.org. Follow Petra Wille: https://Petra-Wille.com. Mentioned in this episode: Claude Code, Vibe coding, Ruby on Rails.

One more quote I loved because it centers autonomy and craft: "It’s a tool in our toolbox. We can decide who on our team has fun with it, wants to do it, wants to contribute." That’s the mindset that sustains both momentum and morale.

Inspired by this post on Product Talk.

May 12, 2026
Taste vs. Evidence in the AI Era: What Product Leaders Must Invest In Now

I just finished listening to "Taste – All Things Product Podcast with Teresa Torres & Petra Wille," and as a product leader shipping AI-powered capabilities at HighLevel, Inc., I wanted to pressure-test the sudden obsession with "taste."

If you're curious, you can listen to this episode on Spotify or Apple Podcasts.

The core question landed perfectly for our moment: Is "taste" the must-have skill of the AI era — or just the latest tech buzzword in a world where AI is eating through design, delivery, and discovery?

Teresa pushes back hard, highlighting how slippery the term can be. "It's just this month's flavor of founder mode." She points out that "taste" is rarely defined, can't be easily taught, and too often becomes shorthand for "my preference trumps yours." Just as importantly, "It's not about your taste. It's about your customer's taste."

Petra adds needed nuance from years in the craft: pattern-recognition is real, and some people do develop sharper product sense over time. As she put it, "I am a strong believer that you develop product sense and taste over time. It's never finished."

Both threads lead back to familiar roots in product: product sense, founder mode, and the enduring myth of the lone visionary. They even grapple with the big question on everyone’s mind—Will AI Eat Taste Too?—and where that leaves product teams navigating GenAI, LLMs for product managers, and evolving product strategy.

Here’s my take. "Taste" can be useful as a personal north star, but it is not a decision system. In my teams, we bias toward evidence: continuous discovery, customer interviews, discovery synthesis with opportunity solution trees, and tight collaboration in product trios. Opinion can start the conversation, but evidence should end it.

Practically, that means investing in the skills that compound: Discovery skills — understanding customers, matching solutions to real needs. Human-to-human interaction skills. Learning to collaborate with AI effectively. Critical thinking and judgment grounded in evidence.

On AI collaboration specifically, we treat GenAI as a force multiplier, not a decider. We prototype with AI to explore breadth, then narrow with qualitative and quantitative signals, ablation-style experiments, and clear success criteria. The bar I hold myself to is simple: taste without evidence is just opinion.

Three lines I underlined from the conversation:

"It's just this month's flavor of founder mode." — Teresa Torres

"It's not about your taste. It's about your customer's taste." — Teresa Torres

"I am a strong believer that you develop product sense and taste over time. It's never finished." — Petra Wille

If you want to go deeper, these references are helpful for sharpening judgment without falling into the "great man" theory trap.

Follow Teresa Torres: https://ProductTalk.org

Follow Petra Wille: https://Petra-Wille.com

Founder mode

Marty Cagan: Founder-Style Leadership

Vercel/v0 CEO Guillermo Rauch on building taste: from Lenny Rachitsky’s Linkedin post

Continuous discovery (Read Teresa’s Everyone Can Do Continuous Discovery—Even You! Here’s How

The "great man" theory

Steve Jobs and the myth of the lone product visionary

Have thoughts on this episode? Leave a comment below and share how your team balances product sense with evidence in the age of AI.

Inspired by this post on Product Talk.

May 5, 2026
Supercharge Claude and Cursor with Amplitude Plug and Play: Your AI Analytics Expert in One Install

I’m excited to share that we’ve brought Amplitude Plug and Play to the Claude and Cursor marketplaces—a lightweight way to infuse your everyday prompts with serious product analytics context and speed.

"Learn more about our new AI plugin, the easiest way to turn your favorite AI client into an analytics expert with a single-install."

For years, I’ve watched teams lose momentum hopping between dashboards, docs, and spreadsheets just to answer simple questions like “What changed in activation last week?” or “Which cohort is driving retention?” With Amplitude analytics and behavioral analytics at the core, Amplitude Plug and Play collapses that friction by bringing the answers to where you already think and build—inside Claude and Cursor.

In practice, this means I can ask natural-language questions such as “Show me the funnel from signup to activation by region,” “Compare retention week over week for new users from our latest release,” or “Summarize our last A/B testing results on onboarding” and get structured, context-aware responses. The goal is to keep me in flow while still honoring the rigor of a unified analytics platform.

What I love most is how this elevates both discovery and delivery. Product managers can accelerate continuous discovery by querying cohorts, drivers, and anomalies mid-conversation. Engineers working in Cursor or with Claude Code can validate event definitions, sanity-check metrics, and spot regressions without leaving their IDE. The result is tighter feedback loops and better decision quality.

Just as importantly, the experience is designed for clarity and consistency. When I ask about activation, I expect the same canonical definition every time. When I explore a retention analysis, I want clear assumptions and transparent logic. By anchoring responses to well-defined metrics and event taxonomies, the plugin helps reinforce good data governance while keeping the interaction fast and conversational.

Getting started takes only a few minutes. Open the Claude or Cursor marketplace, search for Amplitude Plug and Play, complete the single-install flow, and connect to your Amplitude analytics workspace. From there, start prompting as you normally would—only now your AI client can reason with product context.

This launch is part of how I see gen ai reshaping AI workflows for product teams: less context switching, more signal per prompt, and a shared, accessible understanding of what’s really moving the business. If you’re ready to turn your AI assistant into a trusted partner for product insight, Amplitude Plug and Play is a powerful next step.

Inspired by this post on Amplitude – Best Practices.

May 1, 2026
CPO Leadership in the AI Era: A Practical System for Focus
You open a portfolio review and find an AI request from nearly every direction. One team wants an assistant. Another wants an agent. A third has a promising prototype that now needs production funding. Every request sounds plausible, yet approving all of them would spread the company across disconnected experiments.

This isn’t primarily a prioritization problem. It is a leadership-system problem. Your job as CPO is to define the customer advantage worth pursuing, concentrate attention on a few coherent bets, specify the evidence that earns more investment, and make it clear what the company will stop doing. The roadmap should record those choices. It should not make them for you.

Allocate attention before you allocate roadmap space

AI expands the number of things a product team can plausibly build. It does not expand engineering capacity, customer attention, management bandwidth, or the company’s tolerance for operational risk at the same rate. That mismatch is why an orderly backlog can still represent a deeply unfocused strategy.

A prototype adds to the confusion because it compresses the distance between an idea and a convincing demonstration. A good demo shows that a capability may be technically possible under selected conditions. It does not establish that customers will adopt it, that it will perform reliably across real workflows, that its economics will work, or that competitors cannot reproduce it.

Before discussing priority, force each proposed investment through four decisions:
- Customer advantage: What will a specific customer be able to do materially better, faster, or more safely?
- Behavioral outcome: What observable change would show that the advantage matters, such as stronger activation, repeated use, retention, or expansion?
- Business consequence: Which company outcome should move if the customer behavior changes, such as NRR, gross margin, payback, or cost-to-serve?
- Opportunity cost: Which existing initiative, workflow, or commitment will receive less attention if this bet is funded?
The fourth decision is where focus becomes real. If a proposal enters the portfolio without displacing time, money, or executive attention somewhere else, the company has not prioritized it. It has merely added it.

A shared driver tree makes these trade-offs visible. Start with the company outcome. Connect it to the customer behavior that must change, the product lever expected to change that behavior, and the evidence required from the current initiative. If a team cannot draw a credible path through those layers, pause the funding discussion until it can. That is more useful than arguing about whether the item belongs near the top or middle of a feature list.

Your leadership context changes how you create this clarity. In a founder-led company, you often need to influence without becoming deferential: preserve the ambition in the founder’s vision while pressure-testing assumptions with customer evidence, data, and portfolio consequences. Under a hired CEO, the emphasis shifts toward explicit investment theses, capital allocation, and a tighter connection among product, financial, and go-to-market plans.

In either setting, ambition must be more precise than a mandate to become an AI company. Name the customer capability you want to own, the workflow in which it matters, and the durable advantage the company can build around it. Technology is an ingredient. Customer advantage is the strategic claim.

Turn AI feature requests into testable investment theses

A feature request arrives with a solution already embedded in it. An investment thesis keeps the solution open long enough to test whether the opportunity deserves capital. That distinction matters when models, interfaces, and implementation patterns are changing faster than an annual plan can absorb.

Rewrite each material AI proposal using this structure:
<!– wp:list {
April 30, 2026
Unleashing Inbound Sales with AI: My Playbook for Launching and Scaling Sales Agents Fast

Inbound leads shouldn’t wait for a rep’s calendar. When we first launched The Service Agent Blueprint, support leaders finally had a clear AI path. Go-to-market and revenue teams are now facing similar uncertainty, so I’m introducing The Sales Agent Blueprint—a practical map for launching and scaling AI for sales with confidence.

For most sales teams, inbound motions require a lot of manual work. I’ve watched leads pile up in queues, waiting for availability rather than being prioritized by buyer intent. That delay costs meetings, pipeline, and momentum—and it’s exactly where a modern AI Strategy can transform your go-to-market strategy.

Agents can run sales conversations end to end – engaging buyers, qualifying leads, and routing high-intent opportunities to the right team to move prospective buyers forward quickly. Humans will still be involved, but will move their focus to the consultative conversations and higher-value work they did not have time to focus on before. In practice, this shift enables cleaner AI workflows, better conversation design, and a healthier balance between sales-led growth and product-led growth.

The questions many go-to-market and revenue leaders are facing now are where do you start? What should success look like? How do you actually test and deploy these solutions? These are the right questions—and the ones I hear most often when teams weigh build vs buy decisions, evaluation frameworks, and CRM integration nuances.

The Sales Agent Blueprint answers those questions. It’s designed to be a strategic guide for sales, revenue, and AI transformation leaders who want to deploy AI for inbound sales fast, prove value, and build momentum. If you’re aiming for eval-driven development, this will help you define success up front and operationalize it.

What’s inside is simple by design yet deep enough to take you from zero to value. The Sales Agent Blueprint is structured around two tracks that reflect how high-performing teams adopt agentic AI: first, launch for quick wins; next, scale for durable growth.

Coming soon: Sales Agent Blueprint. A sleek, blueprint-inspired teaser with the call to 'Scale it' signals tools, playbooks, and workflows to grow revenue, streamline operations, and scale teams with confidence.

Today, I’m releasing the first part of the Blueprint: “Launch it.” It’s a practical guide for getting your Agent live and seeing real results. You’ll learn how to deploy a Sales Agent that runs inbound sales conversations end to end, engaging buyers, qualifying leads, and routing high-intent opportunities to the right outcome in real time—without disrupting your current CRM integration or pipeline processes.

By the end of the “Launch it” track, you’ll be ready to execute with clarity. Here’s how I frame the essential steps, based on what consistently works in the field.

Understand what a Sales Agent is: Discover why they’re different from chatbots and how they work. Build a business case: Prove the basic economics of AI, decide whether to buy or build, and get the buy-in and budget you need to move forward.

Evaluate an Agent: Learn how to define success, choose the right evaluation criteria, and run a focused, high-impact assessment with our five-step framework.

Deploy with confidence: Build a deployment plan that gets your Agent live quickly to engage buyers at peak intent. Learn what to expect at each stage.

Introducing the Sales Agent Blueprint. This crisp, grid-based graphic spotlights step 1—Launch it—signaling day-one activation for an AI sales agent. Explore the framework and get started at fin.ai/blueprint/sales.

Continuously improve performance: After launch, your Agent becomes a system to manage. We’ll show you how to implement a repeatable process to train, test, deploy, and optimize.

The second track, “Scale it” (coming soon), focuses on the organizational and systems design work that unlocks compounding gains. Launching AI is only the beginning. To unlock its full potential, you need to rewire your inbound sales motion—redesigning the buyer journey, building AI-first systems and ownership models, and rethinking how pipeline is generated and scaled. This is where governance, measurement, and team roles evolve to support sustainable growth.

I’ll be building this Blueprint in public as I navigate the same challenges—sharing what works, what to avoid, and how to accelerate time-to-value without sacrificing quality or trust. If you’re ready to turn intent into revenue with agentic AI, this is your head start.

The Sales Agent Blueprint is live now. Explore the full guide at fin.ai/blueprint/sales and start your “Launch it” sprint today.

Inspired by this post on The Intercom Blog.

April 23, 2026

A Practical Scenario Planning System for AI Product Strategy

Your leadership team wants a firm answer: which AI bets belong on the roadmap, and which ones are expensive distractions? The difficult part is not generating ideas. It is deciding what to fund when customer adoption, interfaces, economics, and enterprise constraints could each develop differently.

A useful scenario plan does not hide that uncertainty behind a confident forecast. It converts uncertainty into conditional commitments: what you fund now, what you preserve as an option, what evidence would change the decision, and what you refuse to scale until the right signal appears.

Frame the exercise around a decision, not the future of AI

Broad questions such as “What will AI look like?” produce interesting conversations and weak strategy. Nobody has to choose anything at the end. Start with a decision that has an owner, a planning window, and a meaningful consequence if the underlying assumptions prove wrong.

A usable decision statement looks like this: “For this customer and workflow, should AI become the primary experience, remain an embedded assistant, or stay in discovery while the existing product carries the outcome?”

Write down five elements before discussing scenarios:

The decision: State the product, investment, or sequencing choice that must be made.
The planning window: Define when the choice needs to be made and when it can be reconsidered.
The expensive assumption: Identify what must be true for the proposed strategy to work.
The reversal cost: Separate choices that can be changed cheaply from commitments involving substantial architecture, hiring, migration, or go-to-market work.
The decision owner: Name the person accountable for changing the plan when the evidence changes.

A scenario belongs in the exercise only if it could change that decision. If two imagined futures lead to the same investment, combine them. More narrative does not create more strategic value.

Next, identify the customer need that should remain valid across the plausible futures. Customers may still need to complete a workflow with less effort, avoid costly rework, understand what the system did, or retain control over a consequential action. Those durable needs become the anchor. They keep an empowered product team focused on outcomes when a particular interface, model, or market prediction stops holding up.

Build scenarios from uncertainties that can change the plan

Do not begin by writing a best case, a base case, and a worst case. That format encourages everyone to treat the base case as the forecast. Instead, find uncertainties that are both unresolved and capable of changing your product choice.

Useful uncertainty prompts include:

Customer behavior: Does repeated AI use spread through the intended market, or remain concentrated among enthusiasts and specialists?
Interaction model: Does AI become the main way customers initiate work, or does it operate inside a familiar interface?
Scope of autonomy: Do customers delegate complete tasks, or accept AI only for bounded assistance and recommendations?
Product economics: Does the value created support the cost and operational burden of delivering the experience?
Enterprise constraints: Do security, privacy, compliance, procurement, and change-management requirements permit the proposed workflow?
Organizational readiness: Can the company evaluate, support, govern, and improve the product after launch?

Select the uncertainties with the greatest decision impact, then push competing possibilities to useful extremes. Extremes expose assumptions that a comfortable middle case can conceal. “Graphical interfaces disappear” and “AI remains an invisible utility” should not be treated as predictions. They are boundary conditions for examining what your product would need in very different environments.

For example, crossing the pattern of customer adoption with AI’s place in the workflow creates the following set of hypothetical futures:

Adoption pattern	AI’s place in the workflow	Plausible future	Decision it tests
Use broadens across intended segments	AI becomes the primary interaction	Customers increasingly start and complete the workflow through AI	Whether to redesign the core experience around an AI-first path
Use broadens across intended segments	AI remains embedded	AI becomes valuable infrastructure inside a familiar product	Whether intelligence, context, and workflow integration matter more than a new interface
Use remains uneven	AI becomes primary for specialists	A smaller group wants an AI-native experience while the broader market retains existing habits	Whether to support distinct experiences instead of forcing one migration
Use remains uneven	AI remains embedded	Bounded assistance improves parts of the workflow without replacing it	Whether focused augmentation is a better investment than broad transformation

The point is not to choose your favorite quadrant. Develop each one far enough to reveal its product implications. For every scenario, describe the target customer’s behavior, the job that still matters, the role AI plays, the constraints that become binding, and the most likely way the strategy fails.

Keep the scenarios plausible rather than theatrical. A future that cannot affect a real decision is entertainment. A future that merely restates the current roadmap is confirmation bias.

Turn each scenario into signals, triggers, and stop conditions

A scenario without observable signals is just a story. A decision-ready scenario tells you what to watch, how that evidence relates to an assumption, and what action follows if the signal appears.

The common mistake is to monitor whatever is easiest to count. Trial starts, demo enthusiasm, and requests from technically confident customers can show interest, but they do not establish broad adoption. Early adopters cannot stand in for the whole market. Segment the evidence so that enthusiasm in one cohort does not silently become a claim about every customer.

Build a signal set that answers distinct questions:

Adoption: Are intended customers returning to the AI workflow after the initial trial, and is use spreading beyond opt-in enthusiasts?
Customer value: Is the core outcome improving through less effort, less rework, fewer avoidable errors, or another measure that matters for this workflow?
Trust and control: How often do customers accept, modify, override, or abandon the result, and what reason do they give?
Enterprise viability: Are security, compliance, procurement, or change-management reviews blocking deployment or narrowing the acceptable use case?
Operational viability: Are reliability, latency, support demand, and cost-to-serve compatible with the value being delivered?
Interface behavior: Do customers initiate work through AI, or invoke AI at specific points inside an established process?

Pair every confirming signal with a disconfirming signal. This prevents the team from collecting only evidence that supports the roadmap it already wants.

Strategic assumption	Confirming evidence	Disconfirming evidence	Product response
AI should become the primary interface	Intended customers repeatedly initiate and complete the core workflow through AI	Customers retreat to the familiar interface for consequential work	Keep the AI-first experience as an option while improving embedded assistance and control
Broader autonomy will create more value	Outcome quality improves while customer intervention and rework decline	Escalations, corrections, or abandonment persist as scope expands	Narrow the delegated task and strengthen evaluation, permissioning, and fallback behavior
Adoption can expand through the target market	Repeated use spreads across intended cohorts and survives enterprise review	Use remains concentrated among specialists or stalls during approval and rollout	Preserve a dual experience and address the blocking constraint before funding broad migration

Write the trigger before launching the bet. A practical format is: “When this signal persists in the target segment and this guardrail remains acceptable, move this investment from an option to a commitment. If this disconfirming signal appears, stop, narrow, or redesign the bet.”

The exact threshold will depend on your product, baseline, risk, and decision cost. What matters is agreeing on it before stakeholders can reinterpret ambiguous results. If nobody can say what evidence would reduce or end the investment, the roadmap contains a belief, not a testable strategy.

Convert the scenarios into a portfolio, roadmap, and sprint choices

Once the scenarios and signals are explicit, separate the portfolio by commitment type. This is where scenario planning becomes operating discipline rather than an occasional workshop.

No-regret bets: Investments that support the durable customer outcome across several scenarios. Depending on the product, these may include better evaluation, permissions, observability, fallback paths, data governance, or clearer measurement. Do not label generic platform work “no regret” unless it supports a named customer outcome.
Option bets: Bounded, reversible work that buys information or preserves a future choice. A prototype, limited workflow, architecture seam, or controlled release can test an assumption without committing the entire product.
Contingent bets: Investments that make sense only after a defined signal appears. Keep the entry condition beside the roadmap item so it cannot become committed work through inertia.
High-regret commitments: Expensive moves that are difficult to reverse, such as a forced workflow migration or a large architecture and hiring commitment. Require stronger support across scenarios before making them.

This creates a roadmap with different funding postures, not a backlog pretending every item has equal certainty.

Roadmap lane	Why it exists	What earns progress	What removes it
Durable outcomes	Advance needs that remain important across plausible futures	Evidence that the customer outcome is improving	The need or outcome no longer matters
Evidence bets	Reduce uncertainty between competing scenarios	A learning milestone tied to a strategic assumption	The assumption is resolved or the evidence cannot affect a decision
Triggered scale	Expand an option after its entry condition is met	The agreed confirming signal appears while guardrails remain acceptable	A stop condition appears or the economics no longer support expansion
Deferred commitments	Preserve ideas that are valid only in a narrower future	A named scenario becomes more plausible through observable evidence	The relevant scenario is disconfirmed

At sprint planning, ask what kind of item is entering delivery. Work should either create durable customer value or buy decision-relevant information. “Build an AI assistant” is an output. “Determine whether target customers will delegate this bounded task while retaining acceptable control” is a learning goal that can change a strategic choice.

For each evidence bet, require the product trio to answer:

Which scenario and assumption does this work test?
Which customer segment must provide the evidence?
What behavior or outcome will be observed?
What result would justify more investment?
What result would stop, narrow, or redirect the work?
Which part of the work remains valuable if the favored scenario is wrong?

This also changes the stakeholder conversation. Replace “Which prediction do you believe?” with “Which commitments are justified across plausible futures, which ones are options, and what signal unlocks the next level of funding?” The latter question makes uncertainty governable.

Review the decision when a trigger fires, a critical assumption changes, or the next expensive commitment approaches. A recurring calendar review can help, but elapsed time alone is not evidence. Keep a short decision record containing the active scenarios, current signals, funding posture, stop conditions, owner, and next decision point.

Key takeaways for your next AI roadmap review

Start with a product or investment decision that could genuinely change. Do not try to describe the entire future of AI.
Build scenarios from unresolved uncertainties with high decision impact, including customer behavior and real-world constraints.
Anchor the strategy in customer needs and outcomes that remain valuable across several plausible futures.
Separate confirming evidence from disconfirming evidence, and segment adoption so enthusiasts do not masquerade as the whole market.
Predefine the signal, guardrail, trigger, and stop condition for every material option bet.
Fund no-regret moves now, use reversible work to buy information, and hold contingent commitments until their entry conditions appear.
Connect every sprint item to either durable value or a decision-relevant uncertainty.

At your next roadmap review, choose the most expensive assumption behind one AI initiative. Write the opposite plausible scenario, identify the customer need shared by both futures, and name the signal that would change the funding decision. If the roadmap still makes sense, the strategy is more resilient. If it does not, you have found the adaptation point before the market finds it for you.

References

Product Talk – Predicting the Future: All Things Product Podcast with Teresa Torres and Petra Wille

April 21, 2026

AI-Native Startup Execution: A Practical Operating System

Your startup can produce impressive demos every week and still learn slowly. If model behavior, customer evidence, and deployment feedback move through separate queues, faster shipping only creates more uncertainty.

If you are deciding how to run an AI-native startup, use one standard: how quickly can your team turn a real customer artifact into an evaluated product change and a measurable customer outcome? That standard should shape your wedge, ideal customer profile, team design, sales motion, and operating cadence.

Build the smallest closed learning loop

An AI-enabled company can add a model to an existing product process. An AI-native company has to organize the process around intelligence itself: the data entering the system, the judgment the model makes, the action produced, the evaluation of that action, and the feedback that improves the next decision.

That makes a long AI feature list the wrong starting point. Begin with the smallest end-to-end loop that proves the product’s central claim. In a security product, for example, the loop might start with suspicious activity, produce a risk judgment, stop or downgrade a threat, and capture enough evidence to assess whether the intervention was correct. A polished dashboard without that closed loop is packaging, not proof.

Use three gates before committing to a wedge:

The customer can quantify the problem. Ask for frequency, severity, operational burden, or another observable consequence in the customer’s own workflow. General concern about AI is not enough.
The wedge can create value within one buying cycle. If proving value requires a broad platform rollout, several integrations, and organizational change across multiple functions, you have probably selected a destination rather than an entry point.
Product use improves your data advantage. Determine what feedback the product will capture, whether you can legitimately use it, how quickly it arrives, and which evaluation or model decision it improves. Data volume alone does not create an advantage.

Failing any one of these gates weakens the loop. Urgent pain without accessible data leaves the model blind. Useful data without a near-term outcome produces an experiment that customers may admire but will not adopt. A fast result that teaches you nothing can become a services engagement disguised as software.

Your first version should feel narrower and less polished than the roadmap in your head. It may require manual review, a rough operator interface, or close founder involvement. That is acceptable when the roughness is visible and recoverable. It is not permission to hide model failure, skip evaluations, or make consequential actions impossible to inspect. Cut breadth before you cut control.

Define completion around the loop rather than the interface. You are done with the first proof when a target customer can supply the relevant input, receive the intended outcome, show whether it helped, and feed the result back into a repeatable evaluation. Everything else competes with learning speed.

Choose an ICP that maximizes learning density

The largest market is rarely the most useful starting market. Your early ideal customer profile should concentrate urgency, usable data, workflow access, and a reachable decision maker. Those conditions allow one customer interaction to improve both the product and the go-to-market motion.

Create a one-page ICP card with four evidence fields:

Urgency: What is happening now that makes the buyer act rather than merely agree that the problem matters?
Data pathway: Which alerts, cases, decisions, errors, or other operational artifacts can enter the product and its evaluation process?
Workflow position: Where will your output appear, who will act on it, and what existing step must change?
Success signal: What observable customer outcome would justify adoption, renewal, or expansion?

Rate each field as high, medium, or low, but require an evidence note beside the rating. A founder’s conviction is not evidence. A buyer who can describe the cost of the problem, provide representative artifacts, identify the operator who owns the workflow, and agree on a success signal gives you something testable.

Keep adjacent customer segments out of the first learning loop when they require different data, integrations, evaluation criteria, or buying logic. A larger pipeline can make progress look healthier while mixing incompatible requirements into the roadmap. The practical test is simple: if two prospects would judge the same model behavior by different definitions of success, they should not share an early product plan merely because they could buy from the same company.

Founder-market fit helps here, but it is a compression mechanism rather than proof of product-market fit. Deep domain experience can sharpen the problem statement, reduce translation with buyers, and establish credibility. It cannot substitute for evidence that customers will change a workflow around the product.

Founder-led sales should therefore operate as a structured discovery system. For every lost or stalled opportunity, record the exact objection, the stage at which it appeared, the evidence offered by the buyer, and the suspected root cause. Do not turn each objection into a feature request. Cluster the objections, identify the two most common root causes, and turn those into experiments within a sprint.

Look beyond compliments when judging early product-market fit. Stronger signals appear when customers escalate a problem to your team, volunteer data that can improve the system, or invite you into the workflow where decisions are made. None is conclusive alone, but each requires the customer to spend trust, attention, or organizational effort. That makes them more informative than enthusiasm during a demo.

Organize the team around evaluation and deployment

A conventional feature organization separates product definition, engineering delivery, model work, implementation, and customer support. That creates handoffs precisely where an AI-native startup needs rapid feedback. Organize durable units around a customer outcome, with the people who can inspect data, change the system, deploy it, and evaluate the result.

The exact titles can vary, but the unit needs four capabilities:

Product judgment: Choose the scenario, define the customer outcome, and decide which uncertainty deserves the next experiment.
Model and data judgment: Design evaluations, diagnose failure patterns, and understand whether a change improves one scenario while damaging another.
Product engineering: Turn model behavior into a reliable workflow with usable interfaces, permissions, integrations, and operational controls.
Frontline deployment: Work safely in live customer environments, resolve implementation friction, and return generalizable learning to the product.

Forward deployed engineers and solutions engineers are especially valuable when the product has to enter complicated workflows. They can shorten the distance between a live failure and a product decision. They can also become an unofficial custom-development queue. Prevent that by attaching three fields to every customer-specific change: the broader pattern it tests, the product owner responsible for reviewing the result, and the condition under which the work will be standardized, stopped, or kept outside the core product.

AI fluency should also change your hiring process. Model vocabulary on a resume tells you little about how someone will reason under uncertainty. Give candidates a work sample built around a realistic failure: model performance changes across scenarios, the data distribution is moving, and customer trust is at risk.

Ask the candidate to define the evaluation, explain the consequences of false positives and false negatives, diagnose a precision-recall tradeoff, and translate the technical result into a product decision. The strongest response will not jump straight to a model change. It will clarify the scenario, inspect the evidence, identify what the current evaluation misses, and explain the tradeoff in terms an operator or buyer can use.

Use the same discipline for founder and leadership disagreement. Write down the principle in dispute, the evidence each side is using, the decision owner, the time box, and the condition that will trigger a review. Debate the principle rather than multiplying competing implementation plans. Once the decision is made, commit until the agreed review point. This preserves speed without pretending that an uncertain decision has become permanently correct.

Run model and business evidence on the same cadence

An AI-native startup needs two connected scoreboards. One shows whether the system is becoming more reliable. The other shows whether customers are receiving enough value to adopt, retain, and expand. Reviewing them separately makes it easy to celebrate a model improvement that does not change customer behavior, or a short-term commercial win built on fragile product performance.

Evidence layer	Question	Measures to inspect	Decision it should inform
Scenario quality	Does the system make the right judgment in each important situation?	Precision and recall by scenario; recurring misclassification patterns	Evaluation coverage, data work, model changes, and acceptable operating thresholds
Change detection	Does the system recognize when its environment is moving?	Drift, data freshness, and time to first signal for an emerging pattern	Refresh cadence, investigation priority, and new evaluation cases
Operational experience	Can the customer use the output without creating a new burden?	Alert-fatigue reduction and latency under load	Workflow design, prioritization, and deployment constraints
Business value	Is reliable behavior turning into durable adoption?	Activation, expansion, and Net Recurring Revenue	ICP focus, onboarding, positioning, and roadmap investment

Do not collapse these measures into one composite score. Aggregation hides the failure mode you need to fix. A model can improve overall precision while degrading a high-consequence scenario. Activation can remain flat even when evaluation results improve because the output arrives too late, lacks an escalation path, or does not fit the operator’s workflow. The broken link between the two scoreboards is often the most valuable finding in the review.

Make the weekly customer debrief evidence-based. Bring actual alerts, misclassifications, escalations, and deployment failures rather than a verbal account of how the customer feels. For each material artifact, record the hypothesis it challenges, the evaluation that can reproduce it, the next bet, and the owner. That written log becomes organizational memory when priorities change quickly.

Use an evaluation gate for every consequential release:

Run the relevant scenario-level evaluations, not only an aggregate benchmark.
Inspect newly introduced failure modes as well as improvements to the target case.
Check latency and operational behavior under the conditions the customer will actually use.
Name the owner of monitoring, escalation, and rollback before deployment.

This is not process for its own sake. In an AI product, observability is the control center for trust. It tells your team whether the model still deserves the authority the workflow gives it. Demo speed cannot compensate for an inability to see, explain, and correct failure in production.

Key takeaways for your next operating cycle

Define the product as a closed loop from customer signal to verified outcome, not as a list of AI capabilities.
Commit to a wedge only when the problem is quantified, value can appear within one buying cycle, and legitimate product use improves the data advantage.
Select the first ICP for urgency, data access, workflow access, and a clear success signal. Keep adjacent segments separate when they require different definitions of success.
Turn founder-led sales into a discovery system by logging every rejection, clustering root causes, and testing the two most common causes within a sprint.
Build durable customer-facing units that combine product, model, engineering, and deployment judgment.
Review scenario-level model evidence and business outcomes together, then investigate the link whenever one improves without the other.

For your next planning cycle, choose one ICP, one end-to-end loop, one scenario-based evaluation set, and one customer outcome. Give a single owner responsibility for showing the connection between them. If the team cannot trace that chain with production evidence, the roadmap is still too broad. Narrow it before adding another feature, segment, or model.

References

Shivam.Consulting Blog – Inside Artemis’ AI vs AI Security War: Hiring at Speed, PMF Signals, and Founder-Led Sales

April 21, 2026

From 70 Employees to Dominance: My Playbook for Hypergrowth, Focus, and Top-Down Goals

Scaling a real-world marketplace from scrappy to dominant takes a different kind of product leadership. Reflecting on Christopher Payne’s decade leading DoorDash as President and COO — growing from roughly 70 employees to the dominant food delivery platform in the US — I’m struck by how much of that success hinged on mastering an atoms-based business while still operating with software-level rigor. As a VP of Product Management, I see the same patterns in my own work: relentless clarity on inputs, a bias for builder-executives, and a cadence that keeps leaders close to product details without becoming bottlenecks.

Running an atoms-based business versus a pure software company forces you to obsess over operational physics: unit economics, quality control, on-time reliability, and dense local liquidity. It’s precisely where traditional “bits” executives can stumble. What’s worked for me is a simple “plate spinning” framework for executive attention: identify the five or six plates that must never stop — customer experience, marketplace health, quality and safety, product velocity, platform reliability, and P&L — then schedule recurring deep dives to keep those plates spinning. If a plate wobbles, I drop in, fix the root cause, re-instrument the inputs, and zoom back out.

Hiring at hypergrowth speed only works when you bias toward a “builder mentality.” I look for executives who run toward fuzzy problems, write clearly, and can prove they’ve shipped value with incomplete information. Prior industry experience can be a liability when you’re reinventing the market; first-principles thinkers outlearn domain experts who try to port yesterday’s playbooks. In executive hiring, I’ve found structured work samples and narrative memos far more predictive than marathon interview loops — companies routinely spend too much time on job interviews and too little time evaluating how candidates think and execute.

Great executives never outgrow the details. Staying close doesn’t mean micromanaging — it means sampling the customer journey and instrumenting the system so you can feel where it hurts. In my own practice, I rotate through frontline touchpoints weekly: support transcripts, NPS verbatims, failed checkout sessions, and reliability dashboards. Small signals often reveal systemic issues. A single ciabatta bread moment — the kind of edge-case substitution that seems trivial — can expose broken handoffs, unclear policies, and misaligned incentives across the marketplace.

Top-down goal setting beats bottom-up when you’re aiming for category leadership. Bottom-up targets tend to regress to comfort; they calibrate to today’s constraints, not tomorrow’s possibilities. I set ambitious, top-down outcomes (not output), frame the non-negotiables, and map driver trees to clarify the input metrics that matter. Then I ask empowered product teams to pressure-test the plan, propose approaches, and own the how. This preserves ambition while unlocking creativity — a practical balance of clarity and autonomy that outcomes vs output OKRs were designed to achieve.

One-size-fits-all management is a myth. Early-stage teams need hands-on coaching and fast decisions; later-stage teams need mechanisms that scale: crisp PRDs, pre-mortems, and operating cadences that separate strategy, planning, and execution. The mark of a high-functioning executive team is not uniform style — it’s high candor, fast escalation paths, and visible commitment after debate. In tough moments, a little charisma goes a long way; in practice, that’s not theatrics, it’s steady optimism, simple language, and consistent follow-through that keeps people moving forward.

The hypergrowth skill stack for executives is surprisingly learnable: ruthless prioritization under uncertainty, narrative writing that aligns cross-functionally, structured delegation with clear “inspection points,” and a weekly rhythm that protects maker time. I leverage a cadence of business reviews (inputs > outputs), customer-scent checks, and decision logs so we can move fast without losing the thread. CEO and executive time management is the ultimate forcing function — if we can’t show where our attention maps to goals, the team won’t either.

Some of my enduring lessons echo the best of Amazon and eBay: customer obsession beats competitor obsession, input metrics beat lagging vanity metrics, and simple mechanisms beat heroics. From Jeff Bezos’s playbook I borrow the insistence on written narratives, single-threaded ownership, and clarity on what will not change. Those principles remain the backbone of platform scalability and resilient product strategy, especially when markets get noisy.

AI is about to flatten organizations. With agentic AI, retrieval-first pipelines, and AI workflows embedded into product development, managers can widen their span without losing fidelity. I see LLMs for product managers accelerating discovery, PRD drafting, and experiment analysis — while raising the bar on decision quality. The implication for leadership: fewer layers, more transparency, and even greater pressure to define sharp, top-down outcomes that teams can autonomously pursue.

If I had to compress this into a playbook, it’s this: set audacious, top-down goals; keep your “plate spinning” calendar sacred; write more than you talk; hire builders, not resume archetypes; sample the customer journey every week; and build mechanisms that make the right thing easier than the heroic thing. That’s how you scale product management leadership from dozens to thousands — in atoms, in bits, and in the messy, exhilarating space where they meet.

April 17, 2026

Tag: AI Strategy

Start with the decision your ROI model must support

Write the measurement contract before you build the dashboard

Instrument the complete journey, not just the conversation

Calculate economic value without turning activity into savings

Count only incremental revenue

Separate capacity from cashable savings

Include the operating costs that make the agent dependable

Do not bury risk inside an average ROI number

Prove incrementality before claiming impact

Turn ROI into a portfolio operating system

Key takeaways

References

Define the product around recovered monomers

Build the AI as a closed learning system

Use stage gates that end in physical evidence

Plan the roadmap backward from cost parity

Key takeaways for your next roadmap review

References

Key takeaways: the playbook on one page

Scale executive judgment before you scale titles

Make IPO readiness a company cadence, not a finance workstream

Allocate attention before you allocate roadmap space

Turn AI feature requests into testable investment theses

Frame the exercise around a decision, not the future of AI

Build scenarios from uncertainties that can change the plan

Turn each scenario into signals, triggers, and stop conditions

Convert the scenarios into a portfolio, roadmap, and sprint choices

Key takeaways for your next AI roadmap review

References

Build the smallest closed learning loop

Choose an ICP that maximizes learning density

Organize the team around evaluation and deployment

Run model and business evidence on the same cadence

Key takeaways for your next operating cycle

References