Your team has access to capable AI. People are producing polished drafts in seconds. Yet the backlog is no shorter, decisions are not arriving earlier, and managers are still checking every important output line by line.

The missing step is not another model upgrade or a company-wide prompt library. You need to turn model output into accepted work: a defined unit that reaches its next recipient faster, meets the same quality bar, and does not create a hidden correction burden downstream.

Stop measuring AI by how impressive its output looks

A convincing answer is not the same thing as completed work. Workplace productivity starts only when an output can pass the acceptance test that already governs the job.

Claude Fable 5.1 and GPT-6 Astra averaged 65% of the rubric items on recurring internal work. In that evaluation, 100% represented an output that was acceptable to automate. The models’ work at 65% would still have been rejected or returned for revision.

That distinction matters. A model that satisfies 65% of a rubric has not automated 65% of the task. It may have completed the easiest portions while missing the judgment, validation, or communication work that determines whether the artifact can be used. If a person must inspect every claim, find the important omissions, and repair the final recommendation, the organization has accelerated generation without necessarily accelerating delivery.

Separate four layers when you evaluate an AI initiative:

• Capability: Can the model produce a plausible response?
• Artifact quality: Does the response satisfy the requirements for the document, analysis, decision, or action?
• Workflow productivity: Does the accepted unit of work move through the process with less total effort or delay?
• Business value: Does the improved workflow create more capacity, better service, lower cost, faster learning, or another outcome the organization actually values?

Most pilots establish capability and stop. Your investment case should be based on the last two layers. If the promised outcome is time saved, measure setup, review, correction, and coordination time as well as generation time. If the promised outcome is more throughput, count accepted outputs rather than drafts created.

Choose the unit of work at the handoff, not the job title

“Make product managers more productive” is too broad to design, evaluate, or automate. A job contains discovery, interpretation, negotiation, judgment, writing, and follow-through. Those activities have different inputs, risks, and definitions of done.

The useful unit is a handoff: an artifact or action produced from known inputs, delivered to a known recipient, under explicit acceptance conditions. That boundary makes both value and failure visible.

Vague ambition	Testable unit of work	Acceptance evidence
Analyze customer interviews	Create a theme table from tagged transcripts, with supporting evidence, contradictions, and unanswered questions	Claims point back to the input, quotations are accurate, and the product manager can choose the next research question
Help prioritize the roadmap	Prepare an option set from approved inputs, showing trade-offs, dependencies, assumptions, and missing evidence	No new commitments are invented, assumptions are labeled, and the decision owner can compare the options
Write the executive update	Convert changes from the systems of record into a decision brief with risks, owner gaps, and requested decisions	Metrics reconcile with their origins, decisions are explicit, and unsupported claims are excluded

Start with work that is frequent, bounded, observable, and reversible. These properties give you enough repetitions to learn, keep the failure surface manageable, and allow a person to recover when the system gets something wrong.

Task definition also affects model performance. AI performed better on clearly specified work such as coding and data analysis, and worse on open-ended pattern recognition and judgment. This is not a reason to avoid ambiguous work forever. It is a reason to decompose it. Let AI prepare evidence, identify gaps, or draft options before asking it to own the judgment that connects them.

For each candidate workflow, write down:

• The event that starts the work.
• The authoritative inputs and where they live.
• The transformation the AI is expected to perform.
• The person or system receiving the result.
• The conditions under which that recipient accepts or rejects it.
• The consequence of an incorrect output.
• The fallback when the AI cannot complete the task.

If you cannot name the recipient and the rejection conditions, the unit is not ready for automation. You are still describing an aspiration rather than a workflow.

Build the definition of done before you tune the prompt

A prompt tells the model what to attempt. A rubric tells the organization what it is willing to use. Write the rubric first so the team cannot lower the standard after seeing an attractive output.

Visual polish is particularly easy to mistake for quality. When given many examples of an organization’s design style, Fable 5.1 produced a crowded chart that failed to communicate clearly. A human designer reduced the information and produced a cleaner result. The model could generate the artifact, but it did not make the editorial judgment that gave the artifact its purpose.

Evaluation must also cover the surrounding system. In one pilot experiment, GPT-6 Astra selected a token budget that prevented 22% of the evaluated runs from returning an answer. It then treated those failures as evidence about the tested models rather than recognizing the setup error. A plausible analysis built on a faulty configuration is still a failed analysis.

For a decision brief, a useful rubric might cover:

• Factual integrity: Every material claim is supported by an authorized input, and conflicts between inputs are exposed rather than silently resolved.
• Completeness: Required fields, dependencies, risks, owners, and open questions are present.
• Judgment boundaries: Facts, assumptions, recommendations, and unresolved decisions are clearly distinguished.
• Actionability: The recipient can identify what decision or action is required next.
• Communication: The artifact is concise enough for its audience and does not bury the decision under generated detail.
• Policy compliance: Sensitive inputs are handled only in approved systems, and prohibited actions remain outside the AI’s authority.

Some criteria should be must-pass conditions. An average score should never cancel out a fabricated customer quotation, an unreconciled financial figure, an unauthorized disclosure, or an invented commitment. Separate defects that make an output unusable from imperfections that a reviewer can reasonably tolerate.

Then run a repeatable evaluation:

1. Collect representative cases from the real workflow, including messy inputs and known exceptions.
2. Record the current human process so you have a baseline for quality, effort, and elapsed time.
3. Run the complete AI-assisted workflow, including retrieval, tools, formatting, review, and write-back.
4. Have the normal recipient grade the output against the prewritten rubric.
5. Classify each failure as a model, context, tool, configuration, workflow, or review failure.
6. Change one part of the system, rerun the cases, and keep the failure log.

This produces something more useful than a prompt leaderboard. It tells you whether to keep the AI as a drafting aid, let it execute under supervision, redesign the workflow, or hand over the task.

Measure the correction loop, then increase autonomy

Calculate net productivity, not generation speed

The right calculation is simple in principle: net time saved equals the baseline human effort minus AI-assisted setup, review, rework, and coordination effort. Generation time is only one component, and it is often the smallest one.

Track a compact set of operational measures for comparable work:

• Human touch time: Active time spent preparing inputs, operating the AI, reviewing the result, and fixing it.
• Cycle time: Elapsed time from the workflow trigger to acceptance by the next recipient.
• First-pass acceptance: The share of outputs accepted without material revision.
• Rework: The type and location of corrections, not just the fact that editing occurred.
• Escaped defects: Problems found after the reviewer accepted the output.
• Throughput: Accepted units completed with the same constrained resource.
• Fallback use: How often people abandon the AI path and return to the previous process.

Compare the normal workflow with the AI-assisted workflow across the same kind of work. Do not use the best demonstration as the baseline. Preserve the quality rubric while measuring speed; otherwise, the team can manufacture a productivity gain by quietly accepting worse work.

Adoption is diagnostic, not proof of value. High use with high rework can mean the tool is easy to access but expensive to trust. Low use with strong evaluated performance can expose a different problem: missing context, awkward handoffs, unclear ownership, or a tool that sits outside the place where work is actually completed.

Connect the workflow before teaching prompt craft

A CPO can run a workday across Slack, Jira, Claude, and PowerPoint without treating prompt engineering as the prerequisite. That does not prove a productivity gain by itself, but it identifies where the implementation work belongs: authorized context, clean handoffs, appropriate permissions, and review inside the tools where people already make decisions.

A clever prompt cannot compensate for stale Jira data, missing customer context, contradictory definitions, or an output that must be copied manually through several systems. The prompt is one interface to the workflow. The operating design determines whether the result can move.

Use evidence to move up the autonomy ladder

1. Suggestion: AI retrieves, organizes, or proposes material while the user constructs the final artifact.
2. Draft: AI creates the artifact, and a person performs a full review before it moves.
3. Supervised action: AI prepares or executes a defined step, but a person approves the consequential handoff.
4. Bounded delegation: AI executes reversible actions inside explicit constraints and routes exceptions to an owner.
5. Task handoff: AI owns the complete unit while monitoring checks quality, drift, and exceptions.

Do not promote a workflow because the model is newer or because employees are enthusiastic. Increase autonomy when must-pass criteria are being met, review effort is falling, failures are visible, a rollback path exists, and an accountable person owns exceptions. Keep approval at the point of consequence when an action creates an external commitment, exposes sensitive information, changes a system of record, or is difficult to reverse.

The transition point is not when AI becomes generally impressive. It is when a particular unit of work reliably clears your organization’s acceptance bar. Basic chart generation reached that point in the real-work evaluation and was removed from further testing because acceptable output could usually be produced in one attempt. That is the pattern to look for: evidence at the task level, followed by a deliberate change in ownership.

Key takeaways

• Define productivity as accepted work delivered with less total effort or delay, not as content generated faster.
• Evaluate a bounded handoff rather than an entire role or a vague promise to make a function more productive.
• Write must-pass acceptance criteria before prompts, pilots, or vendor demonstrations influence the standard.
• Test the whole system: context, tools, configuration, review, exceptions, and write-back as well as the model.
• Count setup, correction, and coordination costs when calculating time saved.
• Increase autonomy only when task-level evidence supports it and the organization can detect and recover from failure.

Start with the next recurring handoff your team already understands. Write its rejection conditions before opening the model, run the complete workflow, and time the full correction loop. If accepted work reaches its recipient faster without creating downstream cleanup, you have a productivity gain worth scaling. If it does not, the failure log will tell you what to fix before the rollout gets larger.

References

• Epoch AI – Can AI automate Epoch?
• Product School – Connect, Build Skills, Delegate: A CPO’s AI Playbook

---