Where is the true boundary between product and engineering—and what happens when it gets blurry? I’ve led and coached teams through this question many times, and I’ve learned that clarity here isn’t just a nice-to-have; it’s foundational to quality, velocity, and team health.
I’ve seen well-intentioned product managers step in to “help” by taking ownership of bug triage, tech debt prioritization, or even system architecture. At first, it feels productive. Over time, it creates role confusion, slows decision-making, and burns out PMs—while paradoxically lowering engineering quality. The “CEO of the product” myth and legacy IT, project-based mindsets are usually at the root. Treating engineers as “order takers” breaks down in evergreen product environments.
The healthiest collaboration model is simple and disciplined: The product trio owns the “what”; engineering owns the “how”. Product managers are not people managers for engineers—and shouldn’t be accountable for engineering quality. Our job is to frame the problem, align on outcomes, and continuously discover value with customers—not to supervise technical execution.
If quality is a problem, the solution is escalating and fixing the system, not managing individual bugs. In practice, that means surfacing patterns and elevating them to engineering leadership, who can address root causes—staffing, skills, code health, CI/CD gaps, observability, or process design—rather than asking PMs to paper over issues with status updates. This keeps accountability where it belongs and reinforces outcomes vs output OKRs.
One high-leverage move is to remove unnecessary intermediaries. Removing the PM as a middleman creates better flow and clearer ownership. Create direct paths for stakeholders to get bug status without routing everything through product. Use dashboards, shared tools, or Slack channels instead of one-off updates. In my teams, shared Jira views, Slack incident channels, and status pages eliminated handoffs, improved stakeholder management, and gave engineers the space to solve problems end-to-end.
Strong engineering leadership is non-negotiable. What strong engineering leadership should own (and why that matters) is the technical system, quality guardrails, sustainable pace, and the practices that uphold them—incident management, code review rigor, test coverage, and SLOs with SRE. Skilled engineering teams naturally push back when boundaries are crossed—and that’s a good thing. It signals ownership, craft pride, and a pathway to durable execution.
When do I step in as product? Primarily to clarify desired outcomes, sequencing, and trade-offs—bringing customer and business context to the table. I structure product roadmapping and sprint planning around value slices and risks, not task lists. I align on decision rights early: architecture and tech debt strategies live with engineering; product strategy, positioning, and success metrics live with product; discovery and prioritization live with the product trio.
Here are the system-level moves I’ve found most effective: Escalate systemic quality issues to engineering leadership, not individual contributors. Advocate for real engineering leadership if your org expects product teams—not IT teams. Then reinforce a culture of continuous discovery so product, design, and engineering make better upstream decisions together. This is how empowered product teams ship higher-quality outcomes—without burning anyone out.
If you’ve ever found yourself acting as the middleman for bug status or being asked to “own” engineering decisions outside your expertise, you’re not alone. Reset the boundaries, make work visible, and double down on shared outcomes. In my experience, the moment we clarify roles and remove status theater, quality rises, cycle time improves, and everyone does the job they were hired to do—better.
“Speed is not the enemy of safety; it is the prerequisite for it.” I live by this principle. In our organization, the average time from merging code to it being used by customers in production is just 12 minutes, and that short window is fundamental to how we build, ship, and learn.
In January 2026, we are averaging 180 ships per workday – roughly 20 deployments every hour. Conventional wisdom suggests that to increase stability, you must slow down. I believe the opposite. Speed is not the enemy of safety; it is the prerequisite for it. Accumulating code creates risk; shipping small batches minimizes it. Shipping is our company’s heartbeat.
Maintaining this frequency while targeting 99.8+% availability has required over a decade of focused investment in systems, principles, and processes. We protect the integrity of our systems through three layers of defense: an automated pipeline that is simple, reliable, and removes the need for manual intervention, a shipping workflow that promotes ownership and uses guardrails as accelerants, and a recovery model that optimizes for mitigating inevitable failures. Here’s how we’ve built each layer so that velocity is our greatest source of stability.
While our platform consists of various services and frontend applications, I’ll focus here on our Ruby on Rails monolith. It is our core application and the one we deploy most frequently; we also deploy it to three different data‑hosting regions with independent pipelines. Our other services follow similar pipeline principles and safeguards, but the Rails monolith is the clearest example of how we ship at scale.
The automated pipeline is designed to move code from merge to production as fast as possible while enforcing strict safety checks. It is fully automated, and the vast majority of releases require no human intervention—critical for CI/CD at high deployment frequency.
Once an engineer merges code to GitHub, two things happen immediately. First, the build: we compile the Rails application and its dependencies into a deployable asset (a slug) in about four minutes. Second, parallel CI: our test suite runs alongside the build; through extensive optimization, parallelization, and test selection, the vast majority of CI builds finish in under five minutes.
As soon as the slug is built, it’s deployed to a pre‑production environment. CI does not block the progression of the slug to pre‑production. Deploying to pre‑production takes around two minutes. This environment serves no customer traffic, but it is connected to our production datastores, mirrors our production infrastructure variants (e.g., web serving, asynchronous worker), and is configured so that requests exercise the pre‑release code and workers.
Immediately after deployment, we run and await several automated approval gates. We verify that the application boots cleanly on hosts (boot test), confirm the parallel test suite passed (CI check), and execute functional synthetics using Datadog Synthetics on critical flows—such as loading or editing a Fin workflow. If any gate fails, the release is halted and does not go to production.
Once approved, we promote the code to thousands of large virtual machines. A deployment orchestrator triggers these deployments simultaneously, while a decentralized, staggered rollout avoids changing the state of the entire fleet at the same millisecond. Within each machine, a rolling restart mechanism removes a process with old code from the serving path, lets it drain gracefully, and replaces it with a fresh process running the new code. From the moment a deployment starts, first requests are served by new code within roughly two minutes, and the vast majority of the global fleet updates transparently within six minutes. When restarts trigger on every machine, production unblocks so the next deployment can begin.
We treat a stalled pipeline as a high‑priority incident. If the automated system rejects three consecutive release attempts, it pages an on‑call engineer. These are pre‑production blocks, but if the shipping lane stops moving, changes pile up—and our stability relies on building and shipping in small steps. The on‑call’s job is to restore flow so that tiny, safe, frequent updates continue to keep risk low.
Our shipping workflow is built on extreme ownership: tools assist, but the engineer is accountable for quality and the decision to merge. I insist that you are present when you ship. The practical benefit of a 12‑minute deployment cycle is that engineers remain in the zone, focused on the problem they just solved, and ready to validate behavior as it goes live.
A rocket lifts into a luminous sky, a metaphor for shipping code fast without breaking things, where precision, automation, and guardrails power 180 safe deployments a day.
To support this, our deployment system sends Slack notifications the moment code is submitted and as it advances through stages, embeds direct observability links to relevant dashboards and logs in every PR and message, and prompts verification so engineers actively watch the dials and test features in production. It is not acceptable to rely on green builds. You’re expected to watch your change go live and if you’re not prepared to rollback, you’re not prepared to ship. We maintain a no‑blame culture: quick rollbacks and immediate reverts are signs of vigilance and ownership, not failure.
We make extensive use of feature flags to turn deployment into a non‑event. By decoupling deployment (moving code to servers) from release (turning features on), we shrink the blast radius of change. Flags can be enabled for all customers, a specific subset, or disabled for everyone in under 60 seconds through our backend UI. Engineers can group flags into beta features and run phased rollouts; we also ensure flags work consistently across non‑monolith applications. In the past three months, we created over 560 flags—and we actively manage them to avoid permanent complexity.
For complex refactors—especially when behavior should not change—we leverage GitHub Scientist, an open‑source experimentation library. It runs candidate logic (new code) in parallel with existing logic (old code) in production, instruments both paths for result and timing comparisons, and keeps existing behavior user‑visible. That means we can iterate on and validate new code under real load without risking the experience, then switch seamlessly when confident.
When engineers need to go deeper before merging, they can generate a slug and deploy it to a virtual machine, detaching a running production host from the serving path and connecting for manual testing. They can also put a pre‑release slug on a serving machine that handles a small percentage of jobs or web requests. Single‑host validation lets us slice observability to those hosts, compare against the main release, and make low‑level changes safer. Staging is a simulation; production is reality. Testing on a single production host validates assumptions with real‑world data without risking the fleet.
Our recovery model starts from a simple principle: stop monitoring systems; start monitoring outcomes. Traditional monitoring tells you if a server is healthy; we care whether customers are healthy. We rely on heartbeat metrics—vital signs that represent the core value our product provides—such as the rate at which messages and comments are created.
Unlike standard uptime checks, heartbeat metrics are binary in spirit. If message send rates dip below baseline, it does not matter if infrastructure dashboards are green. Down is down, and if customers can’t do their job, uptime percentages are irrelevant. By tracking real‑world success rates as a high‑level signal, we catch subtle degradations that traditional alerting either misses or over‑alerts on.
Because we ship in small, incremental steps and maintain previous releases on our virtual machines, our Time to Recover (TTR) is generally very fast. If a heartbeat metric drops or a critical anomaly is detected right after a ship, the system can trigger an automatic rollback, reverting to the release that was running 20 minutes ago—often restoring service before an engineer responds. For complex issues, engineers can initiate a manual rollback through our deployment UI; doing so also locks the production pipeline to prevent further releases while we investigate and remove problematic code.
Resumption of service is not the end. Every incident prompts an incident review, and we don’t just fix the bug. We ask, “How did the machine allow this to happen?” Then we harden the system so it cannot happen again. This loop—fast shipping, fast recovery, rigorous learning—compounds resilience over time.
This operating model aligns to DORA metrics: high deployment frequency, short lead time for changes, low change failure rate, and rapid time to restore service. It’s a CI/CD and SRE‑informed approach that converts speed into a defensive advantage rather than a liability.
Shipping 180 times a day isn’t a vanity metric; it’s a deliberate choice to protect the customer experience. With a 12‑minute window from code to customer, the feedback loop is tight and engineers retain context—and accountability—for the immediate impact of their work. Maintaining this pace requires more than fast CI; it requires judgment, extreme ownership, disciplined use of feature flags, and a recovery model that monitors outcomes. We rely on human expertise, augmented by these layers of defense, to catch issues before they turn into customer pain. We don’t ship fast despite our need for stability; we ship fast to stay in control of change.
I’ve sat in enough finserv contact center reviews to know the pattern: wall-to-wall dashboards, weekly exports, and colorful charts that still leave teams asking, “So what should we do next?” The truth is, more dashboards rarely create better decisions. What we need is digital analytics that translates signals into action—fast, precise, and privacy-safe.
When I say digital analytics, I mean a unified analytics platform that captures real-time behavioral data across voice, chat, IVR, email, and in-app journeys, then operationalizes it for agents, supervisors, and automated workflows. See how real-time behavioral analytics helps finserv contact centers lower costs, improve resolution speed, and deliver better member experiences.
Dashboards tend to be lagging, siloed, and optimized for reporting, not resolving. They spotlight vanity metrics, bury journey-level friction, and rarely surface the “next best action” that actually moves a member request toward resolution. By the time a trend shows up in a weekly readout, the expensive part—handle time, repeat contacts, churn risk—has already accumulated.
Real-time digital analytics flips that script. Instead of passively describing performance, it continuously detects intent, risk, and friction as interactions unfold—then powers targeted responses. For example, it can route high-risk transactions to specialized agents, prompt dynamic guidance during an escalated call, or trigger a proactive message that deflects a repeat contact. In practice, that means fewer transfers, faster resolution speed, and measurable reductions in operating costs.
For finserv specifically, the payoff is immediate. Agent Analytics surfaces coaching opportunities (e.g., where scripts stall or compliance steps get missed). Retention analysis identifies members at churn risk after a negative experience. Journey analytics exposes where authentication fails or balance inquiries overwhelm queues, so you can intelligently deflect to self-service. And when a potential fraud signal appears mid-session, real-time insights can prioritize routing and alerting without sacrificing compliance.
Implementation should be iterative and outcomes-driven. Start by instrumenting the top five journeys that drive the most cost or dissatisfaction (lost card, fraud dispute, loan status, password reset, payment issue). Tie each to clear outcomes vs output OKRs—think first-contact resolution, repeat-contact reduction, containment rate, and average time-to-resolution—so every analytic signal earns its keep. Then activate insights inside the workflow: agent assist prompts, smart routing, and targeted follow-ups that close the loop.
Governance matters just as much as speed. In a regulated environment, privacy-by-design and data governance are non-negotiable. Build data access controls, audit trails, and consent management into your operating model from day one. Align analytics with regulatory compliance requirements to ensure that what you measure and automate is defensible, explainable, and safe for members and the business.
To accelerate learning, pair digital analytics with controlled experiments. Use A/B testing on IVR flows, authentication steps, and post-call follow-ups to quantify what truly reduces transfers and repeat contacts. Define a minimum detectable effect (MDE) upfront so tests are fast and conclusive. Run continuous discovery with cross-functional product trios (operations, data, compliance) to turn insights into shippable improvements every sprint.
On the stack side, focus on connecting systems you already trust. CRM integration ensures that context follows the member, while tools like Amplitude analytics, Pendo, or Intercom can instrument key digital touchpoints. Whether you choose build vs buy, the principle is the same: consolidate signals into a unified analytics platform, then push decisions and guidance back into the tools agents and members already use.
The cultural shift is from reporting to decisioning. Instead of celebrating more charts, celebrate faster resolutions and fewer escalations. Replace static executive reports with alerting and action playbooks. Make it trivial for supervisors to see what changed, why it mattered, and which play to run next. That’s how you convert data into durable operating advantage.
The mandate is clear: stop drowning in dashboards. Move to digital analytics that captures behavior in real time, respects compliance, and powers operational decisions where they matter most—in the member journey. When you do, cost curves flatten, resolution speed climbs, and member trust compounds.
Inspired by this post on Amplitude – Perspectives.
Every quarter, I watch product teams move from gut feel to data-informed decisions—until instrumentation bottlenecks slow them to a crawl. That’s why I’ve become an advocate for codeless analytics: it removes the dependency on engineering sprints for basic event tracking and lets teams answer product questions in hours, not weeks.
We explain what codeless analytics are, why (and how) Pendo supports them, plus responses to the top three myths about low-code/no-code solutions.
Here’s how I frame it with my teams: codeless analytics enables product managers, designers, and customer success to tag features visually, track interactions, and analyze adoption without shipping code. The goal isn’t to replace engineered events; it’s to accelerate discovery, speed up iteration, and reduce context-switching for developers. In practice, this means cleaner prioritization, faster validation of hypotheses, and tighter product-led growth loops.
Why Pendo? In my experience, Pendo’s codeless model shortens the distance from question to insight. Visual tagging makes event setup accessible, in-app guides and product tours let us experiment with onboarding and activation, and governance controls ensure data remains trustworthy across teams. The result is a unified analytics approach where we reserve custom instrumentation for complex logic while using codeless tracking for everyday product questions.
Let’s address the top three myths I hear most often. Myth 1: “No-code is only for simple use cases.” In reality, most decisions we make weekly—feature adoption, path analysis, funnel drop-offs, and retention analysis—do not require custom code. Codeless analytics handles these well, and when we need deeper context (like server-side events), we complement it with engineered tracking. It’s a both/and, not an either/or.
Myth 2: “Codeless data isn’t accurate.” Accuracy comes from governance, not the method. I set clear standards: naming conventions, tagging reviews, ownership, and periodic audits. With disciplined process, codeless tracking yields consistent, decision-grade data. The added benefit is visibility—non-technical stakeholders can validate the instrumentation themselves, reducing misalignment.
Myth 3: “Engineers must instrument everything to scale.” Engineering time is precious; we should spend it on differentiated capabilities, not on routine click tracking. Codeless analytics scales by empowering product teams to self-serve, while engineering focuses on back-end, performance, and edge cases. When paired with a unified analytics platform and clear data contracts, this model scales cleanly across product lines.
For teams adopting this approach, I recommend a simple operating model: define your core product questions up front, tag features aligned to those questions, connect insights to in-app guides for experiments, and measure user activation and retention continuously. Whether you run Pendo alongside Amplitude analytics or within a broader unified analytics platform, the key is to keep the insight-to-action loop tight.
The future of product analytics is codeless because it puts insights where they belong—directly in the hands of the people designing the experience. When we remove bottlenecks, we learn faster, ship smarter, and drive measurable PLG impact. That’s how we turn product analytics from a reporting function into a competitive advantage.
I set out to create a lightweight, high-impact “Pendo Wrapped” experience for our users—and I did it in under 10 minutes with Pendo MCP. As a VP of Product Management, I’m constantly looking for fast, pragmatic ways to turn product insights into moments that drive engagement. This experiment was about transforming raw analytics into a concise, celebratory year‑in‑review that motivates customers to explore more value.
When I say “Pendo Wrapped,” I mean a simple, narrative-style summary of usage highlights: what got adopted, which moments mattered, and where value showed up most clearly. Framed well, that story reinforces product‑led growth by reminding users why they chose us, nudging them toward the next best action, and strengthening activation and retention without heavy development work.
My approach was straightforward: define a clear objective (celebrate milestones and prompt the next step), choose a focused set of metrics (adoption, engagement, and activation), and target relevant segments. Then I layered the narrative on top of existing analytics using in‑app guides and product tours to deliver the experience where it matters most—inside the product.
The reason it took minutes, not hours, is that Pendo MCP let me work with what we already had—segments, saved reports, and proven guide templates—so I could spend time on the story, not the scaffolding. No code, minimal configuration, and a crisp call to action made it feel polished without being heavy.
Increase revenue, cut costs, and reduce risk with Pendo’s Software Experience Management platform. Optimize the entire software experience to drive adoption and improve engagement.
If you want to replicate this quickly, start by selecting one user segment and three metrics that matter to them, write a two‑sentence narrative that connects those metrics to outcomes, and ship a short in‑app guide with a single, purposeful CTA. That’s enough to deliver a personalized year‑in‑review feel and spark immediate exploration—no new infrastructure required.
What surprised me most was how a small, story‑driven touch created outsized alignment across customers and internal teams. It turned analytics into advocacy, reminded our users of the value they’re already getting, and opened the door to deeper adoption. If you’re pursuing product‑led growth, a fast “Pendo Wrapped” is one of the highest‑leverage experiments you can run this week.
Keeping employees informed and engaged isn’t just a communications challenge—it’s a product challenge. When we treat internal tools like products with clear activation moments, measurable outcomes, and continuous discovery, adoption moves from hope to habit. Over the years, I’ve seen small changes in how we onboard, communicate, and measure compound into dramatically higher engagement, better compliance, and faster time-to-value.
“How to improve onboarding, compliance, and internal communications within your employee tools.” That question guides my approach end to end—from the moment someone logs in for the first time to the day they become an expert, championing best practices across their team.
First, I personalize onboarding to accelerate user activation. I map the critical first actions and design a lightweight sequence of product tours and in-app guides that surfaces only what matters right now. Progressive disclosure, clear UX writing, and thoughtful tooltip design reduce cognitive load. I measure time-to-first-value, A/B test checklist microcopy to remove friction, and use Intercom or Pendo to deliver contextual walkthroughs by role, location, and permission level. Amplitude analytics helps me validate that the guided path leads to the intended activation event and sustained usage.
Second, I make compliance effortless and measurable. Instead of long trainings, I embed micro-learnings and policy nudges directly in the flow of work, with just-in-time prompts and short, scenario-based confirmations. I segment by role to avoid alert fatigue and localize where regulations require nuance. Completion rates, quiz accuracy, and time-to-complete are tracked alongside qualitative feedback. When compliance messaging underperforms, I run A/B testing on tone, timing, and format, then iterate until adherence is both higher and faster.
Third, I orchestrate internal communications as lifecycle messaging—not announcements. Employees get targeted release notes, role-specific tips, and in-app reminders aligned to their stage: new, adopting, proficient, or champion. I avoid channel sprawl by making the primary source of truth available in the product, then reinforcing it via email or chat only when necessary. CRM integration and audience rules ensure relevance, while a champions network and office hours create human touchpoints that deepen trust and accelerate adoption.
Fourth, I close the loop with analytics and continuous discovery. I instrument key events and run retention analysis to understand which behaviors predict long-term engagement. I look at cohorts before and after a new guide or product tour, and I compare lift in user activation and feature adoption over 14-, 28-, and 90-day windows. Amplitude analytics provides the behavioral picture; surveys, interviews, and passive feedback widgets explain the why. Together, these inputs power a product-led growth approach for internal tools—observable, repeatable, and improvable.
When teams ask where to start, I pilot one persona, one workflow, and one high-value outcome. I define the activation event, instrument it, launch a single targeted in-app guide through Pendo or Intercom, and A/B test the onboarding microcopy. Two weeks later, I review retention cohorts and completion data, talk to users, and either scale the pattern or iterate. That cadence builds credibility quickly because it ties every communication to a measurable result.
The payoff is tangible: faster onboarding, higher compliance, clearer internal communications, and employees who feel supported rather than overwhelmed. With disciplined messaging, smart instrumentation, and ongoing discovery, we can turn internal tools into catalysts for performance—and transform engagement from a campaign into a culture.
Every week, I watch the cybersecurity landscape shift under our feet. As a VP of Product Management, I’m responsible for building secure, resilient products—and that means understanding how artificial intelligence is transforming the way IT teams defend, respond, and even anticipate attacks.
Learn the ways in which AI is transforming both cybersecurity offense and defense for IT teams.
First, AI supercharges threat detection and prevention. Pattern-recognition models now sift through endpoint telemetry, identity signals, and network flows to surface anomalies in near real time. In practice, that means fewer false positives, faster prioritization, and earlier containment. We’re pairing behavioral analytics with enrichment from our SIEM/EDR stack so analysts get a ranked, explainable view of risk instead of a noisy alert queue—directly improving mean time to detect and laying the groundwork for scalable threat detection and response.
Second, AI accelerates incident response. We’ve embedded LLM-powered copilots into our SOC workflows to summarize alerts, propose next-best actions, and auto-generate draft remediation steps from playbooks. Orchestration then executes routine tasks—isolating endpoints, rotating credentials, updating tickets—while keeping a human-in-the-loop for approvals. To keep this safe, we use privacy-by-design principles, a retrieval-first pipeline for authoritative playbook content, and eval-driven development to measure precision/recall on suggested actions. The result is meaningful reduction in mean time to recover and more consistent incident management.
Third, the offense is getting smarter—and we need to be honest about it. Adversaries use gen AI to craft targeted spear-phishing, deepfake executive voice notes, and polymorphic malware that evades signature-based tools. We counter by red-teaming with AI, deploying deception tech to waste attacker cycles, and hardening identity as the new perimeter (MFA, conditional access, continuous risk scoring). Education matters, too: when employees see how convincing AI-generated lures have become, phishing reports spike and successful compromise rates drop.
None of this works without strong governance. We treat AI like any high-impact capability: rigorous data governance, model access controls, and AI risk management across the lifecycle. We log model prompts and outputs, restrict sensitive data via contextual policies, and continuously test for drift and bias. This is as much an IT leadership challenge as it is a technical one—clear ownership, well-defined runbooks, and regular tabletop exercises make the difference between resilience and chaos.
If you’re getting started, I recommend a focused 90-day plan: identify one high-signal detection use case, one response playbook ripe for automation, and one employee risk area (usually phishing) for immediate uplift. Instrument everything—latency, precision/recall, MTTR—and iterate with a cross-functional group spanning security engineering, SRE, and product management leadership. With disciplined AI strategy and guardrails in place, you can move faster, reduce noise, and stay ahead of adversaries without compromising data or trust.
Protecting customer data is non‑negotiable—and it must coexist with our need for precise product insights. In my role, I frame every analytics initiative, Pendo Agent Analytics included, around measurable outcomes and rigorous governance so we can accelerate growth without compromising trust.
Increase revenue, cut costs, and reduce risk with Pendo’s Software Experience Management platform. Optimize the entire software experience to drive adoption and improve engagement.
To make that promise real, I anchor implementation in privacy-by-design. Practically, that means data minimization, purpose limitation, role-based access control, auditable workflows, and clear retention policies. These are the same standards I expect from any unified analytics platform and the operating guardrails my team applies in partnership with security and legal.
On the product side, I focus Agent Analytics on the behaviors that move the needle: adoption, feature engagement, user activation, and time-to-value. Paired with in-app guides, product tours, and thoughtful tooltip design, insights become timely interventions that drive product-led growth—while staying within our data governance boundaries.
Reducing organizational risk demands discipline. I pair analytics rollout with a documented data map, DPIAs where appropriate, vendor risk assessments, and clear incident management protocols. We align with regulatory compliance requirements and integrate with cybersecurity practices for continuous monitoring and threat detection and response.
I track success through business and trust metrics: higher adoption, stronger retention analysis, fewer support tickets, and cost savings from deprecating low-value features—alongside clean audits and consistent adherence to governance standards. The outcome is a tighter feedback loop, smarter roadmap decisions, and sustained customer confidence.
If you’re evaluating Agent Analytics, start with a controls checklist, define the minimum viable telemetry for your KPIs, validate consent flows, and pilot with a narrow audience before you scale. This approach balances velocity with vigilance, ensuring we harness analytics for impact without sacrificing privacy or compliance.
I spend a lot of time with CIOs and IT leaders who are moving fast on generative AI. The momentum is real, but so are the risks. When AI touches core workflows, data, and customer experiences, we need a clear, pragmatic plan that blends AI Strategy with disciplined product management leadership and IT governance.
Learn about the risks that AI poses to IT teams, and how they can mitigate them.
Here are the four risks I see most often—and the playbook I use to de-risk delivery while preserving speed and innovation.
Risk #1: Shadow AI and data leakage. Teams experiment with unapproved tools, and sensitive data ends up in prompts, logs, or third-party services. Without strong data governance and privacy-by-design, even a small proof of concept can create outsized exposure.
How I mitigate it: start with an AI acceptable-use policy, data classification, and clear guardrails on what can be prompted. Deploy a redaction layer and secrets management before any model call. Favor a retrieval-first pipeline so models reason over vetted internal knowledge rather than raw or personal data. Conduct vendor due diligence and DPAs up front, and centralize audit logs to support regulatory compliance and incident response.
Risk #2: Hallucinations and unreliable outputs. LLMs are probabilistic; they can fabricate citations, numbers, or steps. In customer support and internal operations, this erodes trust and creates rework—especially when teams assume model answers are authoritative.
How I mitigate it: adopt eval-driven development with task-specific test sets, reference answers, and pass/fail thresholds that gate CI/CD. Ground models with retrieval, constrain outputs with schemas, and keep a human-in-the-loop for high-risk actions. A/B testing, error taxonomies, and continuous monitoring turn model behavior into measurable, improvable Web Vitals for AI reliability.
Risk #3: Expanded attack surface. Prompt injection, data exfiltration, supply chain risks in model providers, and insecure connectors can undermine existing cybersecurity controls. Traditional threat models often miss these new interaction patterns.
How I mitigate it: treat AI as a first-class asset in threat detection and response. Implement input/output filtering, allow/deny lists, content moderation, and strict isolation of tools and connectors. Red team prompts and tools regularly, rotate credentials, and codify runbooks with SRE and incident management for fast containment. Apply least privilege to agents, APIs, and vector stores, and monitor for anomalous tool-use.
Risk #4: Compliance, bias, and auditability gaps. As AI scales, questions about explainability, fairness, data residency, and retention move from theoretical to board-level. Without traceability, it’s hard to satisfy audits or respond to regulators.
How I mitigate it: embed privacy-by-design from the first sprint—data minimization, consent, purpose limitation, and retention controls. Maintain model cards, versioning, and lineage for prompts, datasets, and parameters. Centralize audit logs, set policies for high-risk use cases, and run periodic compliance reviews with security and legal. Cross-functional communities of practice keep changes aligned across product, engineering, and IT Leadership.
Operationally, I anchor AI initiatives to outcomes vs output OKRs, use empowered product teams and product trios to balance feasibility, value, and risk, and integrate model changes into CI/CD with quality gates. This creates a repeatable mechanism to ship safely, learn quickly, and scale what works.
If you’re standing up new AI workflows or hardening what you already have in production, this playbook gives you a practical path: drive adoption confidently, protect your data, and stay compliant while maintaining competitive velocity.
The bottom line: AI risk management isn’t a brake on innovation—it’s how we earn the right to go faster.
I rely on product benchmarks to align teams, sharpen strategy, and accelerate outcomes—especially in healthcare, where stakes are high and complexity is real. Over the years, I’ve learned that the right metrics create clarity across product, engineering, compliance, and go-to-market, enabling faster, safer decisions that translate into measurable impact.
Discover exclusive data and strategies from our Product Benchmark Report. Compare the healthcare technology industry’s performance across key product metrics.
When I evaluate a healthcare product’s health, I focus on a few essentials: activation rate and time-to-value for new users, weekly active usage and feature adoption for clinicians and admins, and cohort-based retention analysis to understand whether value compounds over time. I also look at funnel friction (onboarding drop-off, failed setup steps), support load per account, and reliability signals that influence trust—because in healthcare, trust fuels growth.
Benchmarks turn those metrics into context. They help me answer, “Are we good, or just lucky?” By comparing our numbers to industry peers, I can prioritize the few bets that matter, set outcomes vs output OKRs, and guide empowered product teams to focus on the highest-leverage improvements.
Operationally, I instrument products with a unified analytics platform and tools like Amplitude analytics and Pendo to track user activation, feature adoption, and in-product journeys. Pairing that with continuous discovery keeps insights fresh, while A/B testing and clear minimum detectable effect (MDE) thresholds ensure we ship with statistical confidence.
In practice, my playbook for healthcare product-led growth is straightforward: simplify onboarding with targeted product tours and in-app guides, tighten the first-win loop to reduce time-to-value, and eliminate blockers surfaced by behavioral analytics. Then, reinforce the loop with lifecycle messaging, role-specific education, and clear value propositions for clinicians, operations teams, and executives.
Of course, none of this works without strong governance. Data governance and regulatory compliance aren’t just guardrails; they’re growth enablers. Clear audit trails, privacy-by-design, and reliable incident management build the trust that keeps adoption high and churn low.
If you’re ready to benchmark your roadmap against the market, this report gives you the clarity to spot gaps, the language to align stakeholders, and the metrics to execute with precision. Use it to calibrate your product strategy, guide your next set of experiments, and confidently scale what works across the healthcare technology ecosystem.
Inspired by this post on Amplitude – Perspectives.
From a product leadership vantage point, I’ve learned that the fastest path to trustworthy insights and product-led growth runs through the SDKs we put in developers’ hands. When the instrumentation layer is frictionless, data quality improves, teams move faster, and customer value compounds—especially when you’re building on Amplitude analytics.
I collaborate closely with a Senior Software Engineer on the Developer Experience team, specializing in development of Amplitude's Browser SDK. That partnership has reinforced a simple truth: an exceptional developer experience is a growth lever. Streamlined APIs, clear conventions, and resilient client-side telemetry reduce setup time, eliminate common integration errors, and unlock cleaner event streams for retention analysis and user activation.
On the technical front, our shared priorities center on performance, reliability, and privacy-by-design. We optimize for minimal bundle size and zero-regret API ergonomics, while ensuring robust offline queuing, retry logic, and graceful degradation to protect Web Vitals in real-world conditions. CI/CD guardrails, automated schema checks, and backward-compatible versioning keep event contracts stable and predictable as products evolve.
Data governance is a first-class requirement. Consent-aware collection, PII redaction at the edge, and clear controls for regional data routing align implementation with organizational risk tolerances. When teams trust the pipeline, they are more willing to broaden coverage, accelerate experimentation, and make faster, higher-confidence decisions.
The business impact is immediate. Cleaner event taxonomies drive sharper funnel views, enabling tighter A/B testing loops and faster identification of activation drop-offs. With dependable data, product trios can iterate toward the right experience, boosting activation rates, compressing time-to-value, and supporting durable retention analysis without chasing analytics debt.
Great SDKs also multiply the reach of developer evangelism. Strong documentation, copy-pasteable patterns, and pragmatic examples reduce onboarding friction and promote consistent instrumentation across squads. That consistency scales platform scalability, cuts incident noise, and supports reliable DORA metrics—so teams ship frequently without sacrificing quality.
My takeaway is simple: treat Amplitude's Browser SDK as a product surface, not just a technical dependency. Invest in the Developer Experience team, and you’ll find that every improvement pays dividends across experimentation velocity, data trust, and ultimately, product-led growth. When the foundation is solid, everything built on top gets better—faster.
Inspired by this post on Amplitude – Best Practices.
I obsess over the moments that make or break user trust: how fast a page paints, how responsive it feels, and how stable it stays as content loads. Web Vitals are the clearest lens I have to connect those micro-moments to macro outcomes—activation, conversion, retention, and, yes, SEO ranking. Bringing those signals into Amplitude lets me translate web performance into product decisions that move the business.
Now in Amplitude, improve your website user experience and SEO ranking by measuring and taking action on your Web Vitals.
In practice, I focus on the Core Web Vitals—Largest Contentful Paint (LCP), Interaction to Next Paint (INP), and Cumulative Layout Shift (CLS)—and instrument them as event properties so I can segment by page type, device, geography, traffic source, and user cohort. That gives me a single source of truth that aligns engineering performance work with product metrics like activation and revenue, all inside a unified analytics platform.
My workflow is straightforward: I instrument Web Vitals in the client (sampling if needed), stream them into Amplitude, and build dashboards that pair performance distributions with key funnels. I look for thresholds—where a user’s LCP or INP crosses a boundary and their likelihood to convert or retain drops. When I see those cliffs, I know exactly which pages or audiences to target and which improvements unlock the most value.
From there, I run experiments. A/B testing on navigation layout, image optimization, or lazy-loading strategies helps me validate that a performance lift also drives a statistically significant improvement in conversion or retention. Because the analysis lives in Amplitude, I can quickly cohort users by performance experience (for example, “green” vs “yellow” LCP) and quantify how much better experiences translate into business outcomes—reducing the risk of shipping changes that only move a synthetic score without helping users.
SEO benefits are a welcome compounding effect. When I push more sessions into the “good” Web Vitals range, I typically see lower bounce rates, stronger session depth, and better engagement—signals that support search performance. I treat rankings as an outcome of great user experience rather than the goal itself; by improving real-user metrics, I earn durable gains that don’t evaporate with the next algorithm change.
Operationalizing this is crucial. I define product-level service objectives for LCP, INP, and CLS by key page groups, review them in QBRs alongside activation and retention, and set guardrails so performance never regresses during feature velocity. This turns performance into a habit for empowered product teams rather than a one-off initiative.
If you’re starting fresh, begin with a narrow slice: instrument Web Vitals on your top three entry pages, visualize their distributions in Amplitude, and overlay conversion and retention. Within a week, you’ll see where experience degrades for specific cohorts and have a prioritized, testable roadmap for improvement. The fastest path to better UX and growth is making performance visible where you already make product decisions—and that’s exactly what this workflow delivers.
Inspired by this post on Amplitude – Best Practices.
