Your service agent gives the answer that was true before the latest release. Your sales agent promises a capability that is only available on a different plan. The transcript makes each incident look like a model problem. The real break may be earlier: product truth changed, but the agent’s knowledge, retrieval rules, or permissions did not.
Your goal is not a larger collection of documents. It is a controlled path from business truth to customer answer: current information, enough context to apply it, clear limits on what the agent may say or do, and a safe handoff when the answer is uncertain.
Key takeaways
- Organize knowledge around customer decisions and conditions, not around the structure of your internal documentation.
- Keep one shared layer of product truth, then add separate service and sales instructions for how each agent should use it.
- Make agent readiness a release requirement. Changed knowledge, retired answers, permissions, evaluations, and rollback behavior belong in the release plan.
- Test retrieval, answer quality, policy compliance, action selection, and escalation separately. A single accuracy score will hide the failure you need to fix.
- Turn every production failure into a classified backlog item with an owner, an expected answer, and a retest.
Start with customer decisions, not internal documents
Uploading every help-center page, enablement deck, release note, and policy file can reproduce your information sprawl inside the agent. The agent may retrieve a relevant page and still miss the sentence that governs the customer’s situation. It may also find several plausible answers without knowing which one is authoritative.
Begin with the questions that force a decision. Use customer conversations, searches, escalations, sales objections, and failed agent responses to identify them. Write each one as an intent the agent must handle: Can this account use the feature? What should happen when setup fails? Is this customer a fit? Can the agent make this commercial commitment? What information requires a human?
Classify the knowledge required for each intent. The distinction matters because each class should be governed and evaluated differently.
- Product facts: what a capability does, its prerequisites, its limitations, and the terminology customers will encounter.
- Policies: the rule the agent must apply, including eligibility, exceptions, approvals, and prohibited commitments.
- Procedures: the ordered checks or actions that move a customer toward resolution or a qualified next step.
- Conditional decisions: how the answer changes with plan, account state, market, product configuration, customer goal, or rollout status.
- Boundaries: what the agent must not infer, reveal, promise, modify, or attempt without approval.
- Escalations: the trigger, destination, context to collect, and expectation the agent should set with the customer.
Use an answer-ready knowledge object
A useful knowledge object should be small enough to retrieve precisely but complete enough to support a decision. As a working heuristic, use one object for one customer intent or one tightly related decision. Give it explicit fields rather than hiding every condition in a long paragraph.
- Intent: the customer question and common ways it may be phrased.
- Approved answer: the concise response the agent can ground in the record.
- Applicability: the plans, products, account states, markets, channels, or rollout conditions for which the answer is valid.
- Required context: information the agent must retrieve or ask for before applying the answer.
- Allowed action: what the agent may do after answering.
- Limit: what it must not claim, assume, or change.
- Escalation: when to hand off, where to send the case, and what context must accompany it.
- Authority: the approved system of record and the person or function accountable for the truth.
- Lifecycle: draft, approved, active, superseded, or retired, together with the conditions that determine when it applies.
Put critical constraints in metadata that retrieval can filter. A plan restriction buried near the end of a broad page is easy to separate from the answer it qualifies. A plan restriction attached directly to the knowledge object is harder to lose.
I would make one rule non-negotiable: missing context must not become a customer-facing assumption. If the answer depends on account state, entitlement, location, or another live fact, the agent should fetch it, ask for it, or escalate. It should not silently choose the most convenient condition.
Share product truth, but separate the operating playbooks
Service and sales agents should not carry separate copies of the same product reality. That creates drift as soon as one copy changes. Keep a shared layer for names, capabilities, prerequisites, availability, limitations, defined terms, and other facts that must remain consistent.
Then add role-specific layers. Maintaining service knowledge means giving the agent the approved path from a customer’s description of a problem to verification, resolution, or escalation. Maintaining sales knowledge means giving the agent qualification rules, bounded positioning, commercial next steps, and handoff instructions without allowing it to invent commitments.
- Shared product truth: what is true about the product and under which conditions.
- Service overlay: diagnostic questions, account checks, safe actions, remedies, escalation triggers, and the confirmation needed before closing the interaction.
- Sales overlay: qualification criteria, approved positioning, fit and non-fit signals, objection handling boundaries, commercial permissions, and the next human or automated step.
- Control overlay: visibility restrictions, approval requirements, lifecycle status, precedence rules, and behavior when the required evidence is unavailable.
Consider a capability that is available only when an account meets specific prerequisites. The shared object should define the capability, prerequisites, and limitations. The service overlay should tell the agent how to verify entitlement and configuration before troubleshooting. The sales overlay should tell it how to qualify fit and describe the boundary without implying universal availability. If the account context is unavailable, both agents need an explicit fallback.
Keep role instructions as references to shared facts rather than copied prose. When a prerequisite changes, the shared record changes once. The service and sales layers should change only if the operational response also changes.
Define precedence before contradictions reach a customer
An agent needs deterministic rules for records that appear to disagree. Define the order in which it should trust active versus draft content, account-specific state versus general documentation, approved policy versus informal notes, and current rollout scope versus a broad product description.
- Exclude draft, superseded, and retired records from customer-facing retrieval unless a tightly controlled workflow requires them.
- Attach effective conditions to the answer, not only to the page containing it.
- Restrict sensitive commercial, operational, or account information by agent role and channel.
- Require the agent to surface uncertainty when no authoritative record wins.
- Treat a correct escalation as a valid result. An agent that admits a boundary is safer than one that manufactures completeness.
Pricing and entitlement facts may be shared, but sales discount authority and service exception authority are not automatically interchangeable. Access control must follow the decision the agent is permitted to make, not merely the department that created the document.
Make agent readiness a release gate
The stale-answer window opens when product behavior changes before agent knowledge does. The practical control is to make agent readiness part of every product release. The product launch and the knowledge launch should be treated as one operational change.
Add a knowledge-impact brief to the release work. It should answer the questions that a release note alone usually leaves unresolved.
- What customer-visible behavior, terminology, eligibility, limitation, workflow, or commercial claim changes?
- Which service questions, sales questions, agent actions, and handoffs are affected?
- Which plans, account states, markets, channels, or rollout cohorts receive the change?
- What remains unchanged and could be mistakenly generalized?
- Which knowledge objects must be created, revised, superseded, or retired?
- When does each answer become active, and what condition ends its validity?
- Who approves the underlying truth, the service procedure, and the sales use?
- What should the agent do while rollout state or customer context is uncertain?
- Which evaluation cases prove that the old and new conditions are handled correctly?
- What knowledge state must be restored if the product change is rolled back?
Use a definition of ready that tests the whole path
A release is knowledge-ready only when the truth exists, the correct agent can retrieve it, the answer preserves its conditions, the permitted next action works, and the fallback is defined. Merely publishing a page proves none of those things.
- Content ready: affected records are approved, scoped, and active at the correct point in the rollout.
- Conflict ready: obsolete claims are retired or clearly subordinated, rather than left beside the new answer.
- Retrieval ready: representative customer language finds the intended record and excludes restricted or inactive material.
- Response ready: the agent states the answer, limitations, and required context without adding unsupported claims.
- Action ready: service procedures, sales handoffs, and connected workflows behave as the instructions require.
- Escalation ready: the agent knows when to stop, where to route the interaction, and what context to pass.
- Rollback ready: reverting the product also reverts the applicable knowledge and evaluations.
Test both sides of a change. A new feature case should return the new answer for an eligible account, while an ineligible or not-yet-migrated account should still receive the correct old-state response. This is where broad release prose often fails: it explains what is new but not where the previous truth still applies.
After release, inspect interactions associated with the changed intents. Look for customer wording that the evaluation set missed, old terms that still retrieve retired content, missing account context, and handoffs that lack enough information for a human to continue. Feed those cases back into the knowledge object and the evaluation bank.
Measure the failure chain and assign the right owner
A correct document does not prove that the agent will produce a correct outcome. The path runs from customer intent to retrieval, from retrieval to interpretation, from interpretation to an answer or action, and from that action to a customer result. Measure each stage separately.
- Coverage: does an approved answer exist for the intent and its important conditions?
- Retrieval: did the agent find the authoritative active record rather than a related or obsolete one?
- Grounding: does every material claim follow from the retrieved evidence?
- Conditional correctness: did the agent preserve plan, account, market, rollout, and policy constraints?
- Action correctness: did it choose and complete only a permitted next step?
- Escalation quality: did it stop at the right boundary and transfer useful context?
- Freshness: does the answer reflect the product state applicable to this customer?
I would not use a single helpful-answer score as the operating metric. A fluent response can be grounded in the wrong record, correct for the wrong account, or followed by an invalid action. Score the dimensions independently so the repair goes to the right owner.
| Observed failure | Likely failure class | Corrective action |
|---|---|---|
| No reliable answer exists | Knowledge coverage | Create or approve the missing knowledge object and assign a domain owner. |
| Different conversations receive incompatible answers | Conflict or lifecycle | Resolve the authoritative answer, define precedence, and retire superseded records. |
| The right record exists but is not used | Retrieval | Improve intent language, metadata, object boundaries, filters, or indexing behavior. |
| The right evidence is retrieved but the conclusion is wrong | Instruction or reasoning | Tighten the decision rule, add a counterexample, and add the case to evaluation. |
| The answer is correct but the next step fails | Tool or workflow | Repair the integration, permissions, validation, or handoff contract. |
| The agent reveals restricted material or refuses permitted material | Access control | Correct role, channel, audience, and record-level authorization rules. |
| The agent should stop but continues guessing | Boundary or escalation | Add an explicit stop condition, destination, and required handoff context. |
Every production issue should become a reproducible backlog item. Record the customer intent, relevant account or rollout context, agent output, expected behavior, knowledge retrieved, failure class, accountable owner, corrective change, and retest result. Without that structure, teams repeatedly edit prompts for problems caused by stale policy, poor retrieval, missing context, or a broken workflow.
Segment monitoring by agent role, intent family, product area, plan or account state, rollout cohort, knowledge object, and outcome. A blended average can look healthy while a newly released path fails consistently. Prioritize by consequence as well as volume: an uncommon unsupported commercial promise or unsafe account action may deserve attention before a frequent but harmless wording issue.
Do not optimize deflection in isolation. A service agent that escalates a restricted account change and a sales agent that declines to promise an unsupported capability may both be working correctly. Set the acceptable behavior for each risk class before interpreting the metric.
Start with the next product change rather than trying to clean the entire repository. Create its knowledge-impact brief, separate shared truth from service and sales instructions, write the evaluation cases before publication, and require named owners to approve the result. Once that workflow survives a real release, make it part of the product definition of done and expand it to the next high-consequence customer journey.
References
- The Intercom Blog – The new product introduction process: How to make sure your Agent is ready every time you ship
- The Intercom Blog – The ultimate guide to knowledge management for your Service Agent
- The Intercom Blog – The ultimate guide to knowledge management for your Sales Agent








