You have rewritten the system prompt, added examples, and changed models. The demo looks better. Then the application reaches production and still retrieves an expired policy, forgets an approval from three steps ago, or calls a tool that should not have been available.

At that point, another prompt rewrite is unlikely to solve the underlying problem. You need to inspect the complete information environment assembled for the model on that turn. That environment – not the prompt alone – is the product surface you must design, test, and operate.

Stop debugging the prompt when the runtime context is wrong

The practical distinction is straightforward: prompt engineering improves how an instruction is expressed, while context engineering determines what the model can see when it performs the task.

That context can contain instructions, conversation history, saved memory, retrieved records, tool definitions, tool results, and the current state of a multi-step workflow. The model receives a temporary snapshot assembled from those components. It does not automatically know that one policy supersedes another, that an earlier decision has been reversed, or that a database record changed after retrieval. Your application has to make those relationships explicit.

This is why prompt iteration can produce a frustrating pattern: one failure disappears while another appears. The new wording may help the model prioritize one instruction, but it cannot repair missing state, remove an obsolete document, or correct a misleading tool contract.

Use this diagnostic before changing a prompt:

• If the model had the right facts but misunderstood the task, inspect the instruction.
• If the model used a wrong, missing, or outdated fact, inspect retrieval and memory.
• If it behaved as though an earlier step had not happened, inspect workflow state.
• If it chose the wrong action or supplied invalid arguments, inspect tool eligibility, descriptions, and schemas.
• If it followed one rule while ignoring another, inspect instruction precedence, duplication, and context overload.

The distinction matters for ownership. A prompt is usually a text asset. Context is a runtime system involving product rules, application code, data pipelines, search, state management, permissions, and observability. Treating all of that as one long prompt hides the actual architecture.

For every important AI workflow, create a context contract. It should answer five questions: What must the model know? Which system is authoritative for each fact? How fresh must that fact be? Which actions may the model take in the current state? What must never enter or survive in context? If the team cannot answer those questions, the application is not ready for consequential autonomy.

Design the four layers of the context stack as contracts

A useful context architecture separates four layers: instructions, memory and state, retrieved information, and tools. Each layer has a different job and should fail in a visible way.

Layer	Question it must answer	What you should specify	Typical silent failure
Instructions	What is the model expected and permitted to do?	Role, objective, constraints, decision rules, precedence, and completion criteria	Conflicting or vague rules produce inconsistent behavior
Memory and state	What is true about this user, task, and workflow now?	Fact types, ownership, update rules, status transitions, retention, and invalidation	An old summary or conversation turn is mistaken for current state
Retrieved information	Which evidence is relevant and authoritative for this request?	Eligible collections, filters, ranking, freshness, provenance, and conflict handling	A plausible but obsolete document outranks the record that governs the decision
Tools	Which actions are available in this state, and what will each action do?	Eligibility, arguments, prerequisites, side effects, permission checks, and error behavior	The model calls an ambiguous tool or acts before a required approval

Keep stable policy separate from task instructions

Instructions become difficult to reason about when permanent policy, temporary task details, examples, and user content are blended into one block. Separate them by function. Stable rules should define boundaries and precedence. Task instructions should define the current objective and acceptance criteria. Retrieved evidence should remain evidence, not become an instruction simply because it appears later in the context.

Resolve known conflicts before the model call. If one rule says an action requires approval and another example shows the action happening automatically, the context assembler should not ask the model to infer which one wins. Give every instruction bundle a version so a failed run can be reproduced after the text changes.

Model memory as typed data, not an endless transcript

A conversation transcript is evidence of what happened. It is not a reliable representation of what is currently true. Someone may change a preference, revoke an approval, correct a name, or abandon an earlier plan. If every message remains equally prominent, the model has to reconstruct state from contradictory history on every turn.

Separate at least three kinds of information:

• User facts and preferences that may persist across sessions.
• Task state such as status, owner, selected option, outstanding questions, and approvals.
• An event history that records how the current state was reached.

Give each stored fact an owner, update path, and invalidation rule. Keep approvals and other consequential decisions in a canonical record rather than relying on a model-written summary. A summary can make history compact, but it should not silently become the only evidence that an irreversible or externally visible action was authorized.

Retrieve evidence with authority and freshness, not similarity alone

Semantic relevance is only one retrieval criterion. A highly similar document can still be the wrong document if it belongs to another customer, jurisdiction, product version, or effective period. Apply access, tenant, document type, status, and freshness filters before asking the model to reason over the results.

Every retrieved item should carry enough metadata for the application to answer: Where did this come from? When was it last updated? Is it currently effective? What scope does it govern? Is there a more authoritative record? When two eligible records conflict, surface the conflict or route it for resolution. Do not concatenate both and hope the model chooses correctly.

Treat tool definitions as executable product policy

A tool description should tell the model what the tool does, when it is eligible, what arguments are required, what side effects it creates, and how failure is reported. Names such as “update,” “process,” or “handle” are too ambiguous when multiple systems or action types are involved.

Do not expose every tool on every turn merely because the agent could conceivably use it. Build an allowlist from the current user, permissions, workflow state, and task. If an action can spend money, publish content, delete data, contact another person, or change a system of record, enforce the required approval outside the model as well as explaining it inside the context.

The model invocation should be the output of a deliberate assembly pipeline:

1. Resolve the actor, tenant, task, and current workflow state.
2. Load the versioned policy and task-specific instruction bundles.
3. Read the canonical facts and decisions needed for this turn.
4. Retrieve evidence using scope, authority, and freshness constraints.
5. Expose only the tools permitted in the current state.
6. Remove duplicates and lower-value material to fit the context budget.
7. Record a manifest of what was included, excluded, summarized, or truncated.
8. Run the model, validate the result, execute permitted actions, and persist the resulting state transition.

This pipeline turns context from an invisible prompt-building side effect into a component your team can review and test.

Diagnose poisoning, overload, and state drift separately

Three context failures can produce similarly confident outputs while requiring different fixes. Labeling all of them “hallucination” sends the team toward model changes before it has examined the evidence the model received.

Context poisoning: an untrustworthy fact entered the snapshot

Context poisoning occurs when wrong, obsolete, out-of-scope, or otherwise untrustworthy information is treated as valid. The output may look well reasoned because the model is reasoning coherently from a bad premise.

Start with the exact claim that failed and trace it backwards. Was it supplied by retrieval, copied from memory, introduced by a tool result, or inferred from a stale summary? Then fix the admission rule for that context component. Useful controls include source identifiers, effective dates, status fields, authority tiers, access filters, explicit expiration rules, and invalidation when a governing record changes.

The important product decision is not merely how to rank information. It is which information is eligible to influence the decision at all.

Context overload: relevant information is buried in accumulated material

A large context window is capacity, not a requirement to fill it. Adding more history, documents, examples, and tool definitions creates more competition for attention. It also makes failures harder to reproduce because the decisive detail may be surrounded by material that had no reason to be there.

Assign a budget by function before assembling the turn. Reserve space for binding constraints and current task state first. Add the minimum evidence needed to answer the request. Include only the conversation history required to interpret it. Expose only eligible tools. Leave enough room for tool results and the final response.

When the context exceeds its budget, evict information in this order:

1. Exact duplicates and repeated instructions.
2. Completed-turn narration already represented in structured state.
3. Irrelevant examples and tools that cannot be used in the current state.
4. Lower-authority or lower-relevance retrieval results.
5. Older conversational detail that does not affect the current decision.

Do not silently evict current constraints, unresolved decisions, required approvals, or the authoritative evidence supporting a consequential action. If those cannot fit, narrow the task, retrieve in stages, or ask the user to resolve the ambiguity.

State drift: the snapshot no longer matches the world

State drift appears during multi-step work. The agent builds a plan, a tool changes a record, another user updates the same object, or a previously valid assumption stops being true. If the application keeps appending observations without refreshing canonical state, later steps operate on a historical picture.

Represent workflow state explicitly with allowed transitions. Store a version with the state and record which version a model used to make a decision. Before a consequential write, reread the affected record and compare it with the version the plan assumed. If it changed, refresh the context and reconsider the action instead of continuing from the stale plan.

Completed steps should update the task state, not merely add another sentence to the transcript. The next turn needs to know what is true now, what remains unresolved, and which earlier assumptions have been superseded.

Evaluate the context pipeline, not only the final answer

A final answer tells you that a run succeeded or failed. It rarely tells you why. Production diagnostics need a context manifest: a structured record of how the application assembled that run.

The manifest should capture:

• Instruction bundle names and versions.
• The task-state version and the canonical memory keys loaded.
• Retrieved record identifiers, scope metadata, timestamps, and ranking signals.
• Tools exposed to the model and their schema versions.
• Context size by layer, including any summarization or truncation.
• Validation results, tool outcomes, and the state transition produced by the run.

Record enough to reproduce the decision without logging sensitive raw content by default. Access controls, retention limits, and redaction belong in the observability design because context can contain customer data, internal policies, and action credentials.

Your evaluation set should vary the context, not just the wording of the request. Include cases where:

• An obsolete document is more semantically similar than the current one.
• Two records conflict and one has higher authority.
• A required memory fact is missing.
• A critical user correction appears earlier in a long conversation.
• Irrelevant documents compete with the evidence needed for the answer.
• The workflow state changes between planning and execution.
• An expected tool is unavailable, rejects its arguments, or returns an error.
• The user requests an action that is valid in one state but prohibited in another.

Define the expected behavior for each case. Sometimes success means answering correctly. Sometimes it means asking a focused question, refusing an ineligible action, surfacing a conflict, refreshing state, or handing the task to a person.

Track context-specific measures alongside product outcomes:

• Whether claims are supported by eligible evidence.
• Whether the application selected the current authoritative record.
• Whether instructions and approval boundaries were followed.
• Whether tool choice and arguments were valid for the workflow state.
• Whether the application detected and recovered from stale state or tool failure.
• How often users must correct facts the system had already been given.
• Task completion, escalation, latency, and cost for the whole workflow.

A single quality score will hide important trade-offs. An application can produce fluent answers while choosing stale evidence. It can complete more tasks while crossing an approval boundary. Review correctness, action safety, recovery, and business outcome separately.

When a regression appears, replay the failed request with the same model and recorded context. Then change one layer at a time: instructions, state, retrieval, or tools. If the failure disappears after a context correction, preserve that case as a regression test. This is more useful than adding another generic instruction telling the model to be careful.

Key takeaways for your next AI product review

• Ask what the model saw on the failed turn before deciding that the prompt or model is at fault.
• Specify instructions, memory and state, retrieval, and tools as separate contracts with owners and versions.
• Treat the context limit as a budget. Protect current constraints and state; remove duplication, stale history, irrelevant evidence, and ineligible tools.
• Keep canonical workflow state outside the transcript, and refresh it before consequential actions.
• Require provenance, freshness, scope, and authority metadata for information that can change a decision.
• Evaluate missing, stale, conflicting, overloaded, and drifting context cases before expanding an agent’s autonomy.

At your next review, pick one failed production run and ask the team to reconstruct its context manifest. Identify the exact instruction, memory fact, retrieved record, or tool definition that shaped the wrong behavior. Then add an admission rule, state transition, or evaluation case that prevents the same class of failure. That is the point where context engineering stops being an AI concept and becomes an operating discipline.

References

• Exploring AI World – Inside the Stack of Context Engineering

---