============================================================ nat.io // BLOG POST ============================================================ TITLE: A Successful AI Task Is an Accounting Boundary DATE: July 21, 2026 AUTHOR: Nat Currier TAGS: AI, Engineering Leadership, AI Economics, Workflow Design ------------------------------------------------------------ The easiest way to improve AI return on investment is to weaken the definition of success. Count a generated pull request as completed work before anyone reviews it. Count a researched claim when the model produces a citation, not when the source supports the sentence. Count a customer-service task when the agent drafts a response, even if the account remains wrong. Count a scheduled appointment when the interface displays confirmation, not when the calendar and downstream systems agree. The dashboard improves immediately. Nothing about the business has to improve with it. This is the hidden problem inside a metric that is becoming central to AI economics: **cost per successful task**. OpenAI now includes it in a broader scorecard for useful intelligence, alongside useful work, dependability, and value at scale. That is a better direction than comparing token prices alone. But the denominator carries the argument. What, exactly, made the task successful? If success is a model-reported state, a user click, or an unchecked artifact, cost per success is another activity metric. If success is tied to evidence that the intended outcome occurred within its constraints, the same number becomes an accounting boundary. It tells the organization when AI spending can be attached to completed value rather than generated motion. That boundary connects three conversations that are usually separated: engineering's definition of done, operations' definition of an accepted result, and finance's definition of unit cost. [ A cheap attempt can be an expensive outcome ] ------------------------------------------------------------ Per-token price and per-call price are real costs. They are simply too early in the workflow to describe economic value. An AI task can consume model inference, retrieval, browser or API calls, sandbox time, storage, evaluation, human review, retries, correction, and incident handling. The first model response may be the cheapest part of the system. It may also create the most downstream work if the artifact looks finished before it is dependable. Consider an illustrative batch of 100 cases. This is a simplified comparison, not a claim about typical AI performance. | Cost component | Workflow A | Workflow B | | --- | ---: | ---: | | AI attempt cost per case | $0.20 | $0.60 | | Cases accepted without rework | 70 | 90 | | Cases requiring $8 of human rework | 30 | 10 | | AI attempt spend | $20 | $60 | | Human rework spend | $240 | $80 | | Total spend for 100 accepted resolutions | $260 | $140 | | Cost per accepted resolution | $2.60 | $1.40 | Workflow A has the cheaper model attempt. Workflow B has the cheaper accepted outcome. Real systems need more complete numbers, and rework is rarely uniform. The example isolates the denominator problem: optimizing one visible unit can raise the cost of the unit the business actually needs. OpenAI's July 2026 [scorecard for the AI age](https://openai.com/index/a-scorecard-for-the-ai-age/) describes cost per successful task as what it takes to reach the outcome. The word _outcome_ matters. It moves measurement past model activity and toward a state someone can use with confidence. [ Success is a contract, not a feeling ] ------------------------------------------------------------ Teams often discover that they have several incompatible definitions of completion. The model thinks it is done when it emits the requested artifact. The orchestrator thinks it is done when the run terminates without an exception. The product thinks it is done when the user sees a confirmation. Operations thinks it is done when downstream state reconciles. Risk thinks it is done when required evidence and approvals exist. The customer thinks it is done when the problem is actually resolved. All of these can be reasonable checkpoints. Only one should govern the accounting unit. For an AI task, an acceptance contract should identify: - the intended state change or usable artifact, - the constraints that must remain true, - the evidence that proves completion, - the authority required for consequential actions, - the unresolved conditions that force escalation, and - the point at which the outcome can be reversed or corrected. The list is not a universal form. A low-risk summarization task may need only source coverage and human acceptance. A production database change may need tests, policy checks, a change receipt, and rollback evidence. The contract should scale with consequence. Anthropic's guidance on [evaluating AI agents](https://www.anthropic.com/engineering/demystifying-evals-for-ai-agents) gives a concrete version of this distinction. For computer-use tasks, checking that an agent reached a confirmation page is weaker than checking backend state to verify that an order was actually placed. The visible interface reports an event. The backend verifies the outcome. That is the accounting boundary in operational form. [ The denominator can be manipulated ] ------------------------------------------------------------ Once a metric influences budgets and performance reviews, people optimize it. AI task metrics are especially easy to improve on paper because the unit remains unsettled. A team can count retries as separate successful attempts. It can remove difficult cases from the evaluation set. It can classify partial completion as success. It can shorten the observation window so later reversals disappear. It can move human correction into another cost center. It can report generated artifacts while ignoring how many are accepted, maintained, or used. None of this requires deliberate fraud. The system naturally fragments work across tools and teams, while the cheapest telemetry comes from the model gateway. Tokens and run completions are visible. Reviewer effort, silent correction, exception handling, and abandoned output often are not. The result is measurement asymmetry: the organization precisely measures production and loosely estimates acceptance. A defensible metric has to make the denominator harder to game. Define a stable acceptance event. Keep failures and escalations in the population. Attach retries and rework to the originating task. Use a time window long enough to catch the reversals that matter. Segment results by consequence and task type rather than averaging unlike work together. This does not make the metric perfectly objective. Acceptance always contains judgment somewhere. It makes the judgment explicit enough to inspect. [ One task can contain several economic units ] ------------------------------------------------------------ “Research this vendor and recommend whether we should sign” sounds like one task. Economically, it contains several different kinds of work. Gathering public facts is production. Checking that the facts are current is validation. Comparing contractual and technical risk is judgment. Recommending a course is a decision artifact. Signing the contract is an authorized action. Later discovering that a cited security certification had expired creates correction and possibly loss. Counting the whole chain as one model task hides where cost and value entered. Counting every tool call as a task creates the opposite problem: lots of precise units with no business meaning. The useful boundary usually sits where responsibility changes. An artifact becomes reviewable. A result becomes accepted. A decision becomes authorized. An action changes external state. Each boundary should emit enough evidence for the next owner to know what they received. This is also why the same workflow can have more than one valid cost metric: - cost per generated candidate helps tune production efficiency, - cost per reviewable candidate measures how much noise reaches experts, - cost per accepted outcome measures usable throughput, and - risk-adjusted cost per retained outcome captures reversals or failures that appear later. These metrics answer different questions. Trouble begins when the earliest one is presented as the last. [ AI changes the work mix, so the baseline moves ] ------------------------------------------------------------ Even a well-defined acceptance event does not solve every productivity question. AI changes which tasks people choose to attempt. METR distinguishes [uplift on old tasks, uplift on new tasks, and uplift in value](https://metr.org/blog/2026-05-08-task-substitution-and-uplift/). A worker may use AI to complete work that would previously have been too expensive or not worth attempting. That can increase scope without producing a clean reduction in hours. It can also create lots of inexpensive artifacts with little value. This matters for accounting because the pre-AI denominator may no longer describe the post-AI portfolio. “We saved 1,000 hours” assumes the same work would otherwise have happened. “This would have taken a person eight hours” does not establish that anyone would have funded those eight hours or that the result is worth them. Accepted-outcome accounting should therefore compare value and cost within a meaningful task class, then examine how the portfolio changed. It should not turn hypothetical human effort into realized savings by default. The discipline is simple to state and difficult to fake: record what was accepted, what it cost end to end, what it displaced, and what changed because it existed. [ Build the ledger from evidence already produced ] ------------------------------------------------------------ Teams do not need a new analytics platform before they can improve this measurement. Start with one consequential workflow. Give each run a stable task identifier. Record model and tool spend, attempts, elapsed time, validation results, reviewer disposition, rework, and final acceptance state. Link the accepted state to evidence in the destination system rather than trusting the agent's completion message. The minimal record might look like this: | Field | Question answered | | --- | --- | | Task class and consequence | What kind of work is being compared? | | Acceptance event | What observable state counts as success? | | Evidence reference | How can another party verify it? | | Attempts and model/tool cost | What did production consume? | | Automated validation | Which checks passed or failed? | | Human review and rework | How much scarce judgment was required? | | Final disposition | Accepted, corrected, escalated, rejected, or reversed? | | Observation window | How long were later failures attributed back? | Once that record exists, teams can answer better questions. Which task classes are cheap to generate but expensive to accept? Where does review dominate cost? Does a more expensive model reduce rework? Are retries improving success or only hiding first-pass weakness? Which acceptance checks predict later reversals? The ledger is useful before it is complete. Its first job is to expose costs that token dashboards omit. [ Not every useful task needs formal acceptance machinery ] ----------------------------------------------------------------- Exploration has value before it has a testable destination. Brainstorming, aesthetic experimentation, and private sense-making often end when the person doing the work decides the output is useful enough. Forcing those activities through an elaborate acceptance contract would add ceremony without improving the decision. The boundary becomes important when an organization wants to claim repeatable productivity, compare systems, allocate budget, automate consequential work, or reduce human capacity. Stronger economic claims require stronger definitions of success. There is also no universal acceptance metric. A support resolution, code change, research memo, and clinical workflow have different evidence and consequence. The point is not to standardize them into one score. It is to prevent their differences from disappearing inside one convenient number. [ The definition of done is part of the financial model ] --------------------------------------------------------------- AI makes production easier to count because every run emits telemetry. It does not make value easier to define. That work still belongs to people who understand the system: what state should change, what must remain true, what evidence is enough, which failures matter, and who can accept the residual risk. Once those decisions are explicit, cost per successful task becomes a powerful metric. It can connect model selection to rework, evaluation to finance, and operational quality to AI investment. Without them, the metric rewards whatever the system can most cheaply label complete. A successful AI task is not merely where the agent stops. It is where the organization is willing to start counting.