Tag: onboarding

How I’m Readying 11,000 Employees for AI: Role-Specific Training and Human-AI Collaboration

When AI transformation is your mandate at enterprise scale, clarity and pragmatism matter more than hype. My approach to prepare 11,000 employees for AI—with role-specific training, modular design, and human-AI collaboration for better results—rests on three commitments: deliver outcomes tied to real workflows, meet people where they are, and make adoption safer and faster than the status quo.

I start with role-specific training because context beats generic content every time. For product managers, we focus on prompt design for discovery, prioritization signals, and faster hypothesis validation. For engineers, we emphasize code generation quality, test coverage, and secure patterns. For sales and customer success, we build repeatable workflows for research, personalization, and objection handling. Tailoring instruction to each team’s daily work drives confidence, reduces friction, and accelerates time to value.

Modular design is how we scale without sacrificing quality. I break the curriculum into atomic learning units—micro-scenarios, checklists, and in-app guides—that can be remixed into learning paths by role, seniority, and region. This enables just-in-time onboarding, easier updates as gen AI evolves, and localized relevance without reinventing the core. Product tours and embedded nudges reinforce learning in the flow of work, ensuring people practice where the value actually occurs.

Human-AI collaboration is a deliberate practice, not a slogan. We codify co-pilot patterns, checkpoints, and RACI-like ownership so humans remain accountable for outcomes while AI accelerates inputs. Agentic AI is introduced behind guardrails: clear data governance, prompt libraries with approved patterns, verifiable sources, and audit trails. The result is speed and consistency, paired with the trust that leaders and regulators expect.

Change management is where strategy becomes reality. I partner with empowered product teams to co-create playbooks, nominate champions, and sequence rollouts by readiness and impact. We keep a tight feedback loop via office hours, internal communities, and role-based enablement so adoption feels like a product we improve, not a policy we enforce. This is product management leadership applied to culture, not just software.

Measurement keeps us honest. I tie every enablement track to business outcomes—cycle time, win rates, customer satisfaction, and quality—validated through A/B testing where feasible. We monitor adoption, satisfaction, and proficiency, then iterate the content and tooling. When teams see their KPIs move, AI stops being an experiment and becomes part of how we win.

If you’re standing up your AI strategy, start small and specific, ship value fast, and scale through modularity. Role-specific training, modular design, and human-AI collaboration aren’t slogans—they’re a repeatable system for building durable capability across the organization.

Inspired by this post on Amplitude – Perspectives.

October 24, 2025
Stop Trusting Static A/B Test Calculators: Why You Need Dynamic MDE Curves Over Time

After years of running experiments at scale, I’ve learned that the quickest way to stall product momentum is to rely on static A/B test calculators that promise certainty from a single sample size number. Real-world data rarely behaves like those calculators assume, and that gap quietly erodes decision quality, speed, and stakeholder trust.

Read about the issues with current A/B test calculators and why experimenters need to see a range of MDEs over time, not a static sample size

Most calculators hard-code fragile assumptions: a constant baseline conversion rate, balanced traffic allocation, independent and identically distributed sessions, no seasonality, no peeking, no novelty effects, and a fixed-horizon stop. They often use normal approximations that break at low counts and ignore the realities of traffic ramping, SRM (sample ratio mismatch), and mid-test product updates. The result is a deceptively precise sample size that fits the math, not the environment.

In practice, product teams peek, traffic fluctuates by day of week, acquisition mixes shift, and funnel variance changes as users move from click to activation to retention. These conditions make “the” required sample size a moving target, not a constant. Treating a static figure as a guarantee leads to underpowered tests, false confidence, and rushed stops that inflate false positives.

The alternative is to manage Minimum Detectable Effect dynamically. Instead of anchoring on a single number, I plan with a range of MDEs over time—power curves that show what lift we can reliably detect after 3, 7, 14, and 28 days as traffic accrues. This reframes the question from “How big should my sample be?” to “What effect sizes can we detect at each decision point given our forecasted traffic and variance?”

At HighLevel, this approach changed our experimentation culture. For example, an onboarding flow test initially “required” three weeks according to a static calculator. Our MDE-over-time view showed we could detect a meaningful 4–6% lift within a week under expected weekday traffic, but only 8–10% on weekends due to volatility. We set a sequential schedule for interim checks, aligned stakeholders on stopping rules, and made a confident call in nine days—saving a sprint and avoiding a premature rollback.

Implementing dynamic MDEs is straightforward: forecast traffic by day, estimate variance from historical data, and simulate power curves across relevant effect sizes. Layer in sequential testing or Bayesian monitoring to avoid p-hacking, include guardrail metrics (e.g., latency, error rates, SRM), and publish an MDE band that updates as data arrives. This transforms your “calculator” into a living decision tool rather than a one-time estimate.

For teams using a unified analytics platform or tools like Amplitude analytics, it’s simple to automate: generate daily MDE curves, annotate ramp changes and seasonality, and expose a dashboard that tracks detectable lift as a function of time and traffic. Pair this with pre-registered stopping rules and a simple communication routine so stakeholders know exactly when and why you’ll decide.

Beyond top-of-funnel conversion, this mindset is critical for retention analysis and revenue outcomes where effects materialize over weeks or months. Plan MDE bands per horizon—early activation, Day-7 retention, and longer-term LTV—so product discovery and product-led growth bets aren’t prematurely judged on the wrong timeline.

The takeaway is simple: retire the illusion of a one-number sample size. Embrace dynamic MDE curves that reflect how your data actually behaves, make faster and more confident calls, and keep empowered product teams focused on outcomes over outputs. Your experiments—and your roadmap—will move with more speed, less drama, and far better signal.

Inspired by this post on Amplitude – Perspectives.

October 24, 2025
Crack User Drop‑Off Fast: My Step‑by‑Step Amplitude Playbook for High‑Impact Growth

When I see a drop‑off curve flattening our growth, I don’t panic—I get curious. Drop‑off is a signal, not a failure, and with the right workflow it becomes one of the fastest paths to unlocking activation, retention, and product‑led growth.

Understanding user behavior is the foundation of every great product. Here’s how to start doing that with Amplitude.

I start by defining the journey that matters most: the path from first touch to first value. That means choosing a clear activation milestone, articulating the “aha” moment, and writing down the specific questions I need Amplitude to answer—where users hesitate, which segments suffer most, and what behaviors correlate with long‑term success.

Before analysis, I ensure the instrumentation is trustworthy in Amplitude analytics. I align on an event taxonomy, enforce data governance and naming conventions, and attach the right properties (channel, plan, device, role). Clean, consistent data is non‑negotiable—without it, you’re optimizing noise.

Next, I build a simple funnel in Amplitude: sign‑up → verification → setup → first key action. I compare conversion and drop‑off by acquisition channel, device, geo, plan, and cohort. This immediately reveals friction points and clarifies whether the problem is message‑market fit, onboarding, or feature discoverability.

To go beyond the first click, I pair funnels with retention analysis and pathing. I review day‑1/7/30 retention, unbounded retention, and lifecycle stages, then cohort users who hit the “aha” versus those who don’t. The contrast tells me which behaviors predict durability and where a timely nudge can change the trajectory.

Insights only matter if they drive action. I translate each friction point into a targeted onboarding improvement: in‑app guides to nudge setup, product tours that surface the core value proposition, and thoughtful tooltip design at moments of uncertainty. For product‑led growth, I prioritize small, testable changes over wholesale redesigns.

Execution is a team sport. Product trios work with forward deployed engineers and customer support to ship experiments quickly. We schedule them in product roadmapping and sprint planning, and measure impact with shared dashboards in our unified analytics platform. That alignment empowers product teams to move fast without guessing.

If you only have an hour, here’s my quick start: connect your data, define 4–6 events that describe the activation path, build a funnel from sign‑up to first value, segment by new versus returning users, and pick one high‑impact experiment to run this week. Close the loop with lightweight product discovery interviews to validate the why behind the numbers.

Drop‑off isn’t a verdict—it’s a map. Use Amplitude to trace where users hesitate, meet them with timely guidance, and iterate until the journey feels effortless.

Inspired by this post on Amplitude – Best Practices.

October 24, 2025
How I Craft Product Surveys Users Love: Proven Tactics for Actionable, High-Quality Feedback

When I need fast, trustworthy insight into what to build next, I turn to product surveys. Done well, they feel respectful, take minutes, and deliver signal we can ship against. Done poorly, they frustrate users and mislead product teams. Over the years, I’ve refined a simple, repeatable approach that consistently yields high response rates and actionable insights across product discovery, onboarding, and product-led growth motions.

Create effective product surveys that capture actionable user feedback, improve features, and support smarter product decisions.

I always start with the decision I need to make. Am I validating a value proposition, prioritizing a feature, diagnosing friction in onboarding, or measuring retention risk? That clarity shapes everything—who I ask, when I ask, and how I phrase the questions. It also aligns the survey with outcomes, not outputs, so results directly inform product roadmapping and sprint planning instead of becoming a vanity report.

Question design is where UX writing discipline pays off. I keep surveys short (5–7 questions), bias-free, and written in the same voice we use in-app. I mix two or three crisp quant questions (e.g., confidence, usefulness, likelihood to continue) with one or two open-ended prompts to surface the “why.” That blend gives me both trend lines and the qualitative texture I need to make confident trade-offs with stakeholders.

Timing and targeting often matter more than question count. I trigger in-app micro-surveys at meaningful moments—right after a user finishes onboarding, explores a product tour, or engages with a newly released feature. For deeper discovery, I segment cohorts (new vs. power users, retained vs. churning) to avoid muddy averages. The right context earns higher completion rates and more honest feedback.

Trust drives participation. I set expectations upfront: how long it will take, why it matters, and how their feedback will shape the roadmap. I also share back the outcome—what we learned and what we shipped—so users see the loop closing. That simple follow-up builds goodwill and sustains response rates over time.

On analysis, I combine lightweight quant with rigorous qualitative synthesis. I chart response and completion rates, then use thematic coding on open text to spot repeating patterns. Where it helps, I apply gen AI to accelerate clustering and sentiment analysis, then validate the themes manually. Finally, I triangulate with product telemetry in Amplitude analytics to confirm that what users say matches what they do.

The most valuable step is translation: turning feedback into decisions. I map insights to clear problem statements, rank them by user impact and strategic fit, and convert them into opportunities on our roadmap. In planning, I pair these opportunities with success metrics tied to activation, adoption, or retention analysis, so we can measure whether changes actually move the needle.

Surveys aren’t a substitute for interviews, but they’re a powerful complement. They help me spot signals at scale, de-risk bets between cycles, and align cross-functional stakeholders around evidence rather than opinions. When surveys are concise, contextual, and connected to action, users feel heard—and teams ship smarter.

Inspired by this post on Amplitude – Best Practices.

October 24, 2025
Create Irresistible Product Tours Users Love: Boost Onboarding, Feature Adoption, and Satisfaction

I’ve learned that the fastest way to earn user trust is to guide people to value within minutes, not weeks. As a VP of Product Management, I treat product tours as a strategic asset for product-led growth—not a band-aid for unclear UX. When we get them right, new users reach that first “aha” moment quickly, power users discover deeper capability, and support tickets quietly decline.

Learn how to create effective product tours that improve onboarding, feature adoption, and the user experience without overwhelming users.

My starting point is simple: every tour must serve a single job-to-be-done. I resist the urge to teach everything. Instead, I define one outcome (for example, sending a first campaign or inviting a teammate) and design a clear, three-to-five step flow. Strong UX writing does most of the heavy lifting—short, actionable language, consistent labels with the UI, and thoughtful tooltip design that highlights only what’s essential.

I rely on a small toolkit of in-app guides that meet users where they are. A concise welcome modal sets expectations and reiterates the value proposition. A checklist breaks the outcome into bite-sized wins. Hotspots and tooltips provide contextual nudges at the exact moment of need. Empty states teach by doing, showing an example and prompting the next action. Together, these patterns turn guidance into momentum without piling on cognitive load.

Personalization is non-negotiable. I segment tours by role, plan, and intent signal. New admins shouldn’t see the same flow as experienced creators. I trigger guides contextually—after users click into a feature, not on login—and I let them skip, snooze, or revisit the tour from a help menu. Respecting autonomy builds trust and keeps engagement high.

Measurement guides every decision. Before launch, I define success metrics like activation rate, time-to-value, and feature adoption. I instrument funnels with Amplitude analytics to track completion, drop-off by step, and follow-on behaviors (did they invite a teammate or create a second project?). I pair this with retention analysis to see whether guided users come back and expand usage. Then I A/B test copy, step order, and trigger timing until the data—and user feedback—tell a consistent story.

Operationally, I put a product trio—PM, design, and engineering—in charge of the tour experiments and integrate them into product roadmapping and sprint planning. We maintain a style guide for in-app guides and UX writing, so the experience feels native and respectful of the brand. Governance matters: we audit what’s live each quarter to avoid guide sprawl and content conflicts as the product evolves.

There are a few traps I avoid. Long, linear tours that try to teach the entire product almost always underperform. Overlapping tooltips can frustrate power users. And no tour should be a substitute for fixing a confusing flow. When a guide consistently underperforms, I treat it as a product discovery signal to simplify the experience itself.

If you’re getting started, here’s a pragmatic plan I use: pick one high-impact flow tied to activation, define a crisp outcome, draft the microcopy, and build a lightweight in-app guide with a checklist and two or three tooltips. Ship to a small cohort, instrument with Amplitude analytics, and review results after a few days. Iterate fast, roll out broader once you see lift, and continue refining as the product and audience evolve.

Thoughtful product tours don’t just teach; they accelerate confidence. When users feel capable quickly, everything improves—adoption, satisfaction, and long-term growth.

Inspired by this post on Amplitude – Best Practices.

October 23, 2025