In my role leading product strategy at HighLevel, I’ve learned that AI search is one of the most overlooked growth levers in a modern product stack. When we treat every query as a moment to understand intent, reduce friction, and guide users to value, AI search stops being a utility and starts becoming a compounding engine for product-led growth.
"Turn AI search into a growth channel with AI visibility, sentiment analysis, revenue impact, and content recommendations in one place."
That single line has become a practical blueprint for how I operationalize AI Strategy: make what users ask visible, interpret how they feel, quantify what converts, and continually recommend better content. AI visibility tells me which intents we serve well (and where we fail). Sentiment analysis connects experience to emotion. Revenue impact closes the loop with attribution. Content recommendations ensure we don’t just diagnose gaps—we close them.
Under the hood, I anchor this on a retrieval-first pipeline that marries behavioral analytics with a unified analytics platform. This lets me trace the path from query to outcome: how users phrase needs, which results earn clicks, where drop-offs happen, and which experiences correlate with activation, retention, and expansion. With that signal, I can prioritize high-leverage content updates, tune relevance, and decide when agentic AI should step in with guided workflows rather than static results.
Measurement has to be rigorous. I rely on eval-driven development to benchmark intent coverage and answer quality, then confirm impact with A/B testing designed around a clear minimum detectable effect. We test ranking tweaks, prompt variants for LLMs for product managers, and new answer types (short snippets vs. deep dives) to isolate what actually moves activation and Net Recurring Revenue. If it doesn’t change behavior or dollars, it’s noise.
The operating model matters as much as the model weights. Cross-functional product trios pair continuous discovery and journey mapping with a lightweight content audit cadence. The CRO role partners with data science to align search KPIs to revenue goals, and solutions engineering ensures CRM integration and downstream systems reflect what users discover. This keeps the system honest: every improvement is traceable from insight to impact.
Finally, governance and scale are non‑negotiable. Privacy-by-design, clear data governance, and observability protect trust while feature flags and CI/CD let us iterate safely. When the fundamentals are strong, we can confidently expand into richer experiences—like proactive recommendations, in-app guides, and voice AI agent handoffs—without sacrificing reliability or compliance.
If your AI search still feels like a black box, it’s time to turn it into a transparent, revenue-linked growth channel. Make the work visible, measure what matters, and let sentiment and behavior guide the roadmap. The payoff is real: better answers, faster activation, and a content system that learns—and sells—every day.
Inspired by this post on Amplitude – Best Practices.
I’ve learned the hard way that sample size calculators can be both empowering and deceptive. They feel wonderfully precise, but they’re only as trustworthy as the assumptions you feed them. When I lead A/B testing at scale, I treat the calculator as a planning tool, not a verdict—then I systematically validate the assumptions behind it so our decisions stay rigorous and our roadmap stays credible.
At a minimum, most calculators assume you know your baseline rate, your “minimum detectable effect (MDE),” your desired statistical power, and your significance level. They also quietly assume independent observations, clean randomization, stable traffic quality, and a fixed test horizon with no peeking. If any of those break, the “right” sample size can be wildly wrong—and the test conclusions can nudge teams toward the wrong product or go-to-market bet.
Baseline and variance come first for me. I estimate the baseline conversion (and volatility) from recent behavior using behavioral analytics, sanity-check it across key segments, and look for seasonality. Tools like Amplitude analytics help me spot anomalies, bots, or instrumentation drift. If baseline is unstable or highly skewed, I either stabilize it with longer lookbacks or narrow the target segment to reduce noise.
Setting the “minimum detectable effect (MDE)” is where product strategy meets statistics. I work backward from an outcome that actually matters: the revenue, retention, or activation uplift that justifies the opportunity cost of building and running the experiment. If that effect size is implausible given historic lift and variance, I rethink the scope or stack changes into a sequenced set of learning experiments rather than overpromising a single moonshot.
For power and alpha, I default to 80–90% power and a 5% significance level unless the downside risk of a false positive is unusually high, in which case I tighten alpha. I choose one-tailed tests only when we would not act on a negative result and we’ve explicitly pre-registered that decision; otherwise, two-tailed is safer for real-world ambiguity.
Randomization and independence are where many tests quietly fail. I randomize at the user level (not session or pageview), guard against cross-device contamination, and ensure consistent exposure via feature flags. If there’s shared context—say, team-based usage or geographic clustering—I account for it via cluster randomization or acknowledge the inflated variance it can introduce.
Traffic allocation integrity is non-negotiable. I monitor for sample ratio mismatch by comparing observed group splits to the intended allocation and immediately pause if they drift. When SRM appears, the root cause is often instrumentation gaps, eligibility filters applied asymmetrically, or caching layers. Fixing that early preserves trust in every test that follows.
Fixed-horizon math assumes no peeking. If stakeholders need continuous reads, I use sequential testing methods with alpha spending or always-valid approaches designed for ongoing monitoring. If we commit to a fixed horizon, we stay disciplined: no early looks, no midstream metric swaps, no retrofitted hypotheses.
Multiple comparisons can quietly inflate false positives. I predeclare one primary metric to decide, define guardrail metrics to protect experience and revenue, and apply appropriate corrections (for example, controlling the false discovery rate) when testing many variants or slicing results by numerous segments.
Duration and seasonality matter more than most roadmaps admit. I run through full business cycles (at least one complete week for daily patterns, longer for B2B buying rhythms), plan for novelty effects, and watch for behavior settling after initial exposure. If the intervention changes long-run behavior, I extend the measurement window or add a post-test holdout to capture durable impact.
Not all metrics are binomial. For revenue, time-on-task, or heavy-tailed distributions, I confirm variance assumptions, use robust estimators or bootstrapping, and consider variance reduction methods like CUPED to improve power without overextending duration. The calculator’s simplicity should not mask the data’s complexity.
Finally, I connect experimentation to product outcomes. I map hypotheses to a driver tree, ensure each test ladders to activation, retention, or monetization, and document assumptions up front so we learn even when results are null. The result is a culture that respects the math and moves faster precisely because we trust our reads.
Here’s the practical checklist I use before pressing “Start”: validate baseline and variance from recent behavior; set an MDE tied to meaningful business impact; choose power and alpha explicitly; confirm user-level randomization and stable exposure; watch for sample ratio mismatch; align on fixed-horizon vs sequential testing; predeclare a single primary metric and guardrails; run long enough to cover seasonality; use robust methods for non-binomial metrics; and write a brief pre-read so the whole team commits to the plan.
When we honor these assumptions, sample size calculators become sharp instruments rather than blunt ones. You’ll ship fewer misleading wins, avoid costly false negatives, and build a repeatable experimentation engine that compounds learning—and results—over time.
Inspired by this post on Amplitude – Perspectives.
Shipping great products is a game of making high‑quality decisions under uncertainty. In my role leading product management, I’ve seen teams stall when classic methods demand huge sample sizes before we can say anything useful. Bayesian statistics has become my go‑to approach for turning sparse data into clear, decision‑ready insights—especially when traffic is limited or experimentation windows are tight.
Understand Bayesian statistics vs. frequentist methods and learn how Bayesian approaches improve experiment insights with small sample sizes.
Here’s why I rely on it in A/B testing: frequentist methods focus on p‑values and long‑run error rates, which are tough to translate into action. With a Bayesian lens, I can express outcomes as intuitive probabilities—“Variant B has a 92% chance to outperform A”—and use credible intervals to communicate likely ranges of impact. That clarity reduces decision friction and helps the team move faster with confidence.
Bayesian methods shine when sample sizes are small and the minimum detectable effect (MDE) of a frequentist test would be impractically large. I incorporate prior knowledge—historical conversion trends, seasonality, and learnings from related experiments—to stabilize noisy early data. Done thoughtfully, priors improve estimate quality without overfitting; I always run sensitivity checks to ensure the posterior is driven by the data we’re observing, not wishful thinking.
In practice, my workflow is straightforward. I set a prior from historical performance in Amplitude analytics, run the experiment, and update the posterior daily. I track the probability of superiority, expected lift, and a credible interval that the CRO role can rally around. When the probability of a meaningful win crosses a pre‑agreed threshold, we ship. When it doesn’t, we bank the learning and move on—no prolonged debates about p‑values that few stakeholders truly understand.
This approach also strengthens product discovery. By using behavioral analytics and retention analysis as informative priors, I can evaluate early signals from narrower cohorts—new geographies, niche segments, or enterprise accounts—where traffic is scarce. The result is faster iteration in product‑led growth environments, even when a full‑funnel test would take weeks to reach frequentist significance.
Operationally, I treat Bayesian experimentation as part of a unified analytics platform strategy. The same posterior machinery that powers A/B testing can support anomaly detection during releases, quantify risk in phased rollouts, and estimate lift from in‑app guides or product tours. Because results are framed in plain language probabilities, cross‑functional teams make better, faster decisions aligned to outcomes rather than outputs.
A few guardrails keep me honest. I preregister decision rules (stop/go thresholds, guardrail metrics), run prior sensitivity analyses, and document assumptions alongside results. That discipline prevents overconfidence, improves reproducibility, and builds trust with leadership.
If your experiments are bottlenecked by low traffic or you’re tired of waiting weeks for a binary “significant/not significant,” consider a Bayesian upgrade. You’ll get earlier readouts, clearer stakeholder communication, and a repeatable path to compounding learning—without sacrificing rigor.
Inspired by this post on Amplitude – Perspectives.
I keep a running list of product wisdom that sounds great on a slide but quietly sabotages execution. Recently, I revisited that list after a deep conversation with a seasoned CPO from a leading security and compliance platform and reflected on how these lessons show up in my own operating rhythm. What follows is my practical playbook for scaling product organizations without losing speed, quality, or the soul of the product.
Most big-tech veterans struggle when they leap into startups because the safety net of process disappears. At a startup, the buck truly stops with you—there’s no committee to shield a decision and no process to rescue a weak plan. The mindset shift is simple to say and hard to do: own outcomes end to end, reduce your reliance on institutional scaffolding, and make decisions with incomplete information while keeping standards high.
“Great product leaders stay in the details.” I sample artifacts every week—PRDs, design flows, user research notes, postmortems—and I read customer threads to calibrate my intuition. To maintain shipping velocity as headcount grows, I instrument a few critical indicators (deployment frequency, change failure rate) and favor outcomes over output. Data guides my attention; it never replaces judgment.
As teams scale, I use a blunt rule to keep speed high: small autonomous teams, small batch sizes, short feedback loops. One clear owner, one prioritized backlog, and weekly demos to customers. We ship thin slices, not big bangs. And “Great CPOs should avoid comfort metrics”—the easy dashboards that rise when nothing meaningful is moving. I push for outcome-centric OKRs tied to customer value, not vanity charts.
Rigid hierarchies derail quality decision-making. They slow signal, encourage escalation theater, and suppress the truth from the edges. I shorten paths between PMs, engineers, designers, research, and go-to-market leads, and I strip out stage gates that don’t add learning. Above all, I refuse to “Stop making your team fetch rocks”—randomized executive requests without context. Instead, I frame clear problem statements, explicit constraints, and observable success criteria.
Revenue and product can feel at odds, but they don’t have to be. The key to a quality CPO and CRO relationship is a shared operating model: one customer narrative, a joint pipeline of problems worth solving, and a common scorecard. We meet weekly, review the same signals, and align on sequencing: what we solve now for impact, what we stage for scale, and what we sunset to reduce complexity. When trade-offs get tough, we anchor on customer value and long-term defensibility.
Who ultimately oversees the quality bar? I do—and I do it through clarity, exemplars, and consistent feedback loops, not micromanagement. When I leave feedback, I make it actionable and specific: name the user scenario, note the friction, propose a sharper decision frame, and suggest a smaller, testable slice. I expect narrative memos and crisp acceptance criteria; I offer rapid, detailed responses so momentum never stalls.
Open office hours are my forcing function for transparency and speed. Anyone can bring a thorny escalation, a design in progress, or a customer insight. Pair that with weekly 1:1s—non-negotiable for developing leaders and unblocking work—and the organization learns to surface issues early, make faster decisions, and self-correct without drama.
Here’s a glimpse into my working week: Mondays set priorities and confirm the few decisions that matter; midweek is for deep reviews across roadmap, research, and engineering readiness; Thursdays I’m with customers and partners; Fridays I write and synthesize. I leave space for unscripted time with individual contributors—because ICs are the unsung heroes of a company—and I celebrate excellent craft out loud.
The hardest leadership skill is knowing when to push and when to give space. I push on clarity, sequencing, and quality; I give space on solutions and implementation paths. I reject comfort metrics, reinforce outcomes vs. output, and keep the organization close to customers and details. If you’re stepping from big tech into a startup or scaling your product org through rapid growth, these practices will help you ship faster, decide better, and raise the quality bar without burning out your team.
I’ve led product organizations through multiple growth chapters, and the pattern is always the same: the tighter the alignment between product, sales, and marketing, the faster you scale. Reflecting on the journey of Chris Degnan — the first sales hire at Snowflake who spent 11 years helping scale the company from zero to $3.5 billion in revenue as its CRO while partnering with four different CEOs — I’m struck by how consistently the fundamentals win. The playbook isn’t mysterious; it’s disciplined execution, ruthless clarity, and a go-to-market strategy that matures with each revenue stage.
At $10M ARR, the CRO role is hands-on and founder-adjacent. You’re close to the product, running point on key deals, pressure-testing messaging, and building credibility with early customers. By $1B+, the job is organization design: segmentation, international expansion, forecast accuracy, enablement, recruiting, and cross-functional orchestration. The shift is from deal quarterback to system architect — standing up repeatable, auditable processes that produce reliable outcomes across regions, segments, and industries.
Sales leaders who can’t sell the product themselves don’t last. Whether you sit in product management leadership or run the field, you need to master discovery, speak the customer’s language, and translate use cases into value. That also means getting fluent in solutions engineering — understanding integrations, data paths, security, and the operational realities buyers live with. I’ve found this hands-on competence to be the fastest way to earn trust internally and externally, and to keep product strategy grounded in market truth.
The MEDDIC methodology is the foundation for every durable sales org — and, frankly, a founder’s best insurance policy. MEDDIC forces alignment on qualification criteria, from Metrics to Economic Buyer to Decision Process and Identifying Pain. When product and sales both operate to this standard, roadmap bets improve, marketing targets sharpen, and win rates climb. It’s not paperwork; it’s pattern recognition at scale.
High-output CROs obsess over the right numbers. Pipeline coverage by segment and stage; conversion rates through each gate; sales cycle length by use case; average selling price and discount discipline; consumption predictability when you have consumption SaaS pricing; and post-sale expansion velocity. The art is deciding which two or three metrics are the organization’s true north at a given stage — then designing enablement, compensation, and operating cadence around them.
On operating cadence, the week in the life at scale is predictable for a reason. Forecast reviews that surface risk early. Deal reviews that coach to MEDDIC depth, not activity theater. Enablement blocks to uplevel managers and ICs. Recruiting time — always. Customer roadshows to refine value proposition and product positioning. And standing meetings with product, marketing, and finance to keep the GTM motion, roadmap, and unit economics in sync.
Compensation is a force multiplier or a silent saboteur. Keep it simple, consistent, and aligned to the current motion. Early on, weight new logo acquisition and land quality; as you mature, balance new business with expansion, multi-product adoption, and healthy consumption. Guardrails matter — cap over-discounting, reward multi-threading, and avoid plans that create end-of-quarter cliff behavior. The best plans reinforce the behaviors you want your culture to scale.
Technical CEOs often underestimate how much narrative, segmentation, and process discipline great GTM requires. The handoff from founder-led GTM to sales-led growth is where many teams stall. My rule: prove one repeatable motion in one segment before you add complexity. Codify the buyer’s journey, instrument the funnel, and make sure product strategy and enablement move in lockstep.
Culture sets the ceiling. You have to find the fakers, manage-uppers, and passengers quickly — people who look busy but don’t move pipeline, who talk big but avoid accountability, or who ride the momentum of others. The mantra that has saved me endless time: “When there’s doubt, there’s no doubt”. Move fast, but with humanity; be clear on expectations, coach hard, and when it’s not a fit, make the change before the team does it for you.
Feedback is the operating system of a high-performing org. Leaders at every level need to be coachable — on message discipline, on forecast rigor, on how they develop people. I’ve benefited from straight talkers who hold a high bar, and I try to pay that forward. The fastest way to raise organizational IQ is to institutionalize feedback loops across sales, product, and marketing — from post-mortems to win-loss analysis to field-sourced roadmap reviews.
What separates exceptional ICs from the rest? Hunger, intellectual honesty, and a builder’s mindset. They qualify hard, align to customer metrics early, multi-thread to power and value, and partner tightly with solutions engineering. They don’t hide from gaps; they surface them, and they know exactly what they need from product, marketing, and leadership to win.
Executive teams that scale share a few traits: crisp segmentation decisions, single-threaded ownership for outcomes, and healthy conflict that resolves into commitment. Dysfunction, by contrast, looks like metrics roulette, opaque decision-making, and a tolerance for exceptions that become precedent. Make the rules explicit and the exceptions rare.
Leaders like Frank Slootman have popularized intensity, speed, and focus — and there’s real power there when paired with clarity and data. The lesson I carry forward: move fast on people decisions, keep the message simple, and measure what matters. Equally important is knowing where that approach can backfire — when speed outruns learning, or when pressure erodes cross-functional trust. The best operators balance urgency with systems thinking.
Most AI companies will face a go-to-market reckoning. Model quality won’t save a weak motion. The winners will articulate a hard-nosed ROI, solve specific workflow pain, address data governance and security head-on, and show measurable lift — not demo dazzle. In other words, the same fundamentals apply; the stakes and scrutiny are just higher.
If you’re building or rebuilding your revenue engine, start here: define your ideal customer profile and segmentation with ruthless clarity; adopt MEDDIC and teach it across product and sales; align compensation to today’s motion; instrument the funnel and inspect it weekly; and cultivate a culture where feedback is fuel. Do that, and the path from $0 to $3.5B stops feeling like mythology — and starts looking like math.
I’ve been reflecting on why so many revenue leaders are at risk of falling behind, and the conclusion is stark: fewer than 10% of current CROs will thrive by 2028. That isn’t hyperbole—it’s a wake-up call for how quickly go-to-market strategy, organizational design, and AI-driven execution are evolving. From my seat leading product, I see the pressure building on the CRO role to orchestrate the entire revenue system, not just run a sales team.
One story that crystallizes this reality comes from the journey of Stevie Case, the CRO of Vanta, the trust management platform serving everyone from founders to Fortune 100 CISOs. A former pro-video gamer who stumbled into sales through a mentor’s bet, she exemplifies how unconventional paths can drive unconventional insight. Her trajectory underscores a bigger truth I’ve witnessed across companies: the best revenue leaders aren’t just great sellers—they’re builders who understand product, process, and people at scale.
Why do early revenue hires fail? In my experience, it’s rarely about raw talent. It’s about fit, scope, and time horizon. Early-stage teams often hire coin-operated closers to sprint for this quarter’s number, when what they actually need are long-term builders who can shape ICP clarity, pipeline math, and repeatable motion. The trap is simple: you hire for momentum before you’ve validated the motion. That misalignment shows up at 00:00 Why early revenue hires fail and again at 04:16 Coin-operated sellers vs. long-term builders—two ideas every founder-led GTM team should internalize before the first half-dozen sales hires.
What separates a VP of Sales from a top 1% CRO is scope and systems thinking. A true CRO owns the full revenue engine—marketing, sales, solutions engineering, customer success, pricing, channels, and post-sale activation—not just the new-business line. It’s a role defined by precision around 07:44 Metrics, confidence, and velocity and the courage to decide when to centralize vs. decentralize capabilities as you grow. Should CROs lead sales? At 12:04 Should CROs lead sales?, the nuance is clear: yes, if the motion is still coalescing; not necessarily, once the machine is humming and specialization unlocks scale. My rule of thumb: start consolidated for speed of learning; split functions only when interlocks are provably robust.
There’s a humbling lesson in 16:36 Learning to scale at Twilio and 19:58 Stevie’s scaling mistake at Vanta: copying another company’s operating system, even a world-class one, is an easy way to blunt your edge. Context is king. What worked at Twilio won’t automatically work at a trust management business. That’s why the line at 17:44 “There is no CRO playbook” resonates so deeply. There are principles—org design, segmentation, enablement, compensation, customer activation—but your playbook must be bespoke to your product, pricing, cycle time, and buyer power map.
22:16 Why Vanta stays 100% sales-led is a reminder that not every high-growth motion demands product-led growth. In categories where compliance, security, and risk shape buying behavior, a consultative, sales-led approach builds trust and shortens time to value—especially when solutions engineering, onboarding, and customer success are tightly choreographed. I’ve seen teams chase PLG headlines while ignoring the higher-ROI path right in front of them: nailing the sales-led experience, from first touch to first value.
Top CROs plan 24–26 months ahead. 23:16 The value of planning 24-26 months ahead isn’t about creating perfect forecasts; it’s about designing optionality. That means hiring with stage gates, building enablement before you feel “ready,” instrumenting activation and retention early, and pressure-testing your pricing and packaging quarterly. In my org reviews, I push for scenario modeling: what breaks at 2x volume, what centralizes again at 600 headcount, and what competencies must be grown vs. bought.
On judgment and decision quality, 29:54 When trusting intuition was the wrong call is a familiar leadership tax. Pattern recognition is powerful—until it isn’t. I’ve learned to pair intuition with a data backstop and a lightweight pre-mortem: what would have to be true for this to fail? It’s the same posture I take with AI in GTM. At 30:49 Do humans still have a place in the future of GTM? and AI vs. humans in go-to-market, the answer is yes—but augmented. Humans set narrative, negotiate ambiguity, and build trust; AI accelerates research, writing, discovery, and coaching. The winning motion fuses both.
I’m often asked which tools materially shift outcomes. For revenue intelligence and operational rigor, I look to systems that compound learning: Gong: https://www.gong.io/, Salesforce: https://www.salesforce.com/, and Cursor: https://cursor.sh/. To study benchmark operating models and developer-led growth infrastructure, Twilio: https://www.twilio.com/ remains instructive. And to understand why trust, security, and compliance can define the entire GTM architecture, Vanta: https://www.vanta.com/ is a useful case study.
Leadership non-negotiables matter more as you scale. 33:33 Stevie’s leadership non-negotiables reminded me to be explicit about standards: clarity over activity, customer outcomes over internal wins, and auditability over anecdotes. 36:36 The myth of hiring for industry expertise shows up again and again—I’d rather hire for learning velocity, systems thinking, and builder DNA than narrow domain familiarity. And at 40:00 What stays centralized in a 600-person company, remember: centralize what must be consistent (data, tooling, pricing guardrails, core enablement), decentralize what benefits from speed and context (segment plays, partner motions, field marketing).
If you prefer a structured digest, here’s the operating checklist I use with revenue and product peers: define your ICP and value proposition crisply; hire builders over coin-operated sellers; instrument the first 30 days post-sale (47:09 The hidden leverage of a customer’s first 30 days); align pricing, packaging, and onboarding to activation; model capacity and hiring plans on 24–26 month horizons; decide early what stays centralized; use AI to amplify discovery, coaching, and content while keeping humans front-and-center for trust-building; and cultivate an unvarnished CEO–CRO pact (01:02:30 Unpacking the CEO-CRO dynamic) that aligns on strategy, segmentation, and sequencing.
For those who want a few timeline highlights: 00:00 Why early revenue hires fail; 02:23 Who to hire at $5M in revenue; 05:57 What excellence looks like in the CRO role; 17:44 “There is no CRO playbook”; 22:16 Why Vanta stays 100% sales-led; 23:16 The value of planning 24-26 months ahead; 47:09 The hidden leverage of a customer’s first 30 days; 53:42 Why the CRO role will face enormous changes by 2028; 58:42 What leaders must do now to stay relevant.
The throughline is simple and urgent. 53:42 Why the CRO role will face enormous changes by 2028 isn’t a forecast—it’s a present-tense mandate. 58:42 What leaders must do now to stay relevant: build a revenue system, not a sales team; plan further out while executing faster; let AI handle the mechanical so your people can master the human. Those who internalize this shift will be the fewer than 10% of current CROs who thrive by 2028. The rest will be outpaced by change they could have anticipated—and designed for.
