Most AI slop is unresolved workflow ambiguity made fluent.
Imagine the world's best table saw in a beautiful workshop. The cast iron is flat. The motor is powerful. The blade is sharp. Its transparent red guard is still installed, exactly where it should be.
Now look around it.
The fence was never squared. The tape measure is missing. The plans are torn, incomplete, and contradictory. One stack of material contains cabinet-grade hardwood. The next contains warped boards, rotten pallets, and workshop garbage. Nobody agreed on the dimensions. Nobody inspected the cuts.
At the end of the day, the crew presents a doghouse. One wall has perfect joinery. The other is scrap. The roof does not fit. The entrance is on top.
The owner points at the saw and says, “This tool produces slop.”
The doghouse is bad. The diagnosis is worse.
This is how much of the current argument about AI goes. We isolate the model from the objective, requirements, context, tool choice, plan, execution environment, validation, and feedback loop that shaped its output. Then we treat the final artifact as a clean measurement of machine intelligence.
It is not. It is a measurement of a system.
The useful question is not whether AI slop exists. It is where the unresolved decisions entered the system, and how to catch them before release.
If you are accountable for AI-assisted work, you can use that diagnosis to find missing decisions and make them testable.
Thesis: Most AI slop is unresolved ambiguity across a workflow, collapsed into a fluent answer and released without sufficient validation.
Why now: Models improved faster than the operating practices, specialist tools, and evaluation systems around them.
Who should care: Engineers, founders, product leaders, operators, and anyone accountable for AI-assisted work.
Bottom line: Do not ask only whether the model is capable. Engineer the conditions under which capability becomes a dependable result.
Key Ideas
- “Averages ambiguity” is an operating metaphor. A model does not calculate the mathematical mean of several meanings.
- Missing decisions widen the Interpretation Surface, the materially different outcomes still compatible with a request.
- CALIBER turns the diagnosis into a workflow: contract the outcome, assemble context, route the work, separate stages, and test the result.
Slop is real. The diagnosis is incomplete
In 2025, Merriam-Webster selected “slop” as its word of the year, defining the modern sense as low-quality digital content usually produced in quantity by artificial intelligence. The word works because everyone recognizes the product: generic prose, interchangeable illustrations, shallow listicles, broken code, invented facts, and decks that look complete before anyone has decided what they mean.
The label names a real cultural experience. It does not identify one technical cause.
Models do fail on their own terms. They hallucinate. They miss instructions. They retrieve the wrong part of a long context. They can converge on familiar phrasing and flatten creative variety. Product teams also make it cheap to generate far more material than anyone can responsibly inspect. Some slop exists because a model failed. Some exists because an incentive system rewards volume over judgment.
But the common diagnosis stops too early. Outcome quality also depends on objective clarity, relevant context, model capability, appropriate tools, staged execution, explicit checks, and informed iteration. That is not a formula with measured weights. It is an accountability map for finding where quality entered or left the system.
Fluency makes the problem harder to see. A broken table warns you by wobbling. A weak AI artifact can arrive with headings, citations, clean syntax, and an executive tone. The surface says “finished” even when the decisions underneath were never made.
A model cannot infer the missing blueprint
A large language model generates likely continuations from learned patterns and the context available now. When a request leaves a decision open, the system has three broad options: ask for clarification, abstain, or choose a plausible default.
Most everyday AI interfaces strongly reward the third option. Users expect an answer. Evaluation systems often reward correctness more than calibrated abstention. Research on why language models hallucinate argues that common training and evaluation incentives can reward guessing when uncertainty would be more honest.
Consider “Make this strategy better.” Better for whom? More rigorous, more concise, more persuasive, more cautious, more original, more practical, or more likely to survive legal review?
| Unresolved dimension | Plausible interpretation A | Plausible interpretation B |
|---|---|---|
| Audience | Board members deciding capital allocation | Engineers deciding implementation sequence |
| Objective | Secure approval | Expose execution risk |
| Evidence | Directional market narrative | Auditable primary sources |
| Time horizon | Next-quarter action | Three-year capability build |
| Tone | Decisive and compressed | Exploratory and qualified |
| Success | Readers agree | Readers can make the next decision |
Every row permits a different good answer. The prompt is not short of adjectives. It is short of decisions.
I call the remaining set of materially different valid outputs the Interpretation Surface: all the different things a request could reasonably mean. Clear decisions make it smaller. Missing decisions make it larger.
“Averages ambiguity” is a metaphor for what the output can look like when the system fills those gaps with learned and contextual defaults. It does not mean a neural network literally computes an average meaning. Nor does it mean every uncertain detail multiplies error by a fixed amount. The practical point is simpler: ambiguity across several dimensions creates more plausible paths, and a fluent default can hide which path was chosen.
A 2025 study of prompt underspecification found that models often guessed missing requirements, and that underspecified prompts were substantially more likely to regress than fully specified versions in the evaluated tasks. Importantly, the researchers also found that naively adding requirements could hurt. More words are not the same as a better contract.
Adjacent research finds sensitivity to meaning-preserving prompt formatting changes, weaker use of information buried in the middle of long context, and limited ability to identify when clarification is needed.
The lesson is not “write a longer prompt.” It is “design an interaction that can resolve missing decisions.”
The artifact changes the result
The most effective way to reduce ambiguity is often not another sentence. It is an artifact that makes decisions inspectable.
Suppose a leader asks: “Make a board deck about our AI strategy. Keep it concise and compelling.”
The request sounds reasonable. It contains almost no operating contract.
| Before input | After output |
|---|---|
| “Board deck” | Ten slides for the September capital-allocation meeting |
| “AI strategy” | Decision between internal platform investment and three vendor-led pilots |
| “Concise” | Twelve-minute presentation, maximum 40 spoken words per slide |
| “Compelling” | Evidence sufficient to approve a $1.2 million bounded experiment |
| Unnamed sources | Finance forecast, security review, pilot results, and dated vendor evidence |
| No review rule | CFO validates economics, CISO validates risk, owner resolves conflicts |
The second column is not a better prompt. It is an outcome contract. The team can challenge the decision, evidence cutoff, scope, and approvers while changes are cheap. The contract can live in a brief, ticket, rubric, structured file, or test fixture, as long as people can debate it before generation and the workflow can evaluate it afterward.
The artifact narrows the Interpretation Surface by moving decisions out of memory and into shared state. It also gives a reviewer something stronger than taste. “I do not like this deck” becomes “Slide six presents an estimate as a verified fact and fails the contract.”
“Small” was not a requirement
In production, I learned the cost of that missing contract in a WebRTC project. A client said the system only needed “small group calls,” so we implemented a mesh topology. In a mesh, each participant connects directly to every other participant. It is simple and effective at very small scale.
The client's definition of small was 12 people. At that size, every machine had to encode one stream and decode eleven. Users' computers were overwhelmed.
The code could be coherent while the architecture was wrong for the intended scale. The ambiguity entered through one ordinary adjective. “Small” sounded like a constraint, but it did not specify the number that determined the topology.
An outcome contract would have forced that decision into shared state before implementation: maximum simultaneous participants, target devices and network conditions, and a measurable threshold for an acceptable call. The architecture and scale test could then be chosen against that ceiling instead of against a demo that only proved a call connected.
AI did not create this failure mode. It shortens the path from a vague requirement to polished implementation. A coding agent could produce the mesh implementation quickly. It still could not decide what the client meant by “small.”
Ambiguity enters before generation
So far, the deck and WebRTC examples each turn on one missing decision. The same problem can enter at every stage.
Prompt engineering treats the instruction as the main control surface. Workflow engineering treats the instruction as one stage in a larger system.
flowchart LR
A[Objective] --> B[Requirements]
B --> C[Context]
C --> D[Model and tool routing]
D --> E[Plan and execution]
E --> F[Evaluation]
F --> G[Release or revision]
A1[Outcome contract] -. reduces ambiguity .-> A
B1[Constraints and examples] -. reduces ambiguity .-> B
C1[Trusted source set] -. reduces ambiguity .-> C
D1[Capability match] -. reduces ambiguity .-> D
E1[Stage boundaries] -. reduces ambiguity .-> E
F1[Tests and reviewers] -. reduces ambiguity .-> FThis is a conceptual workflow map, not a measured causal formula.
Ambiguity can enter at each stage.
The requested artifact can be named while the decision it must support remains unclear. Sources can conflict or go stale. The selected model may not handle the reasoning depth or modality, and the workflow may lack the browser, code runner, spreadsheet, or design surface the job requires. Requirements can collide without a priority rule. If the artifact has no tests, evidence standard, or named reviewer, a hidden default can survive because the result sounds finished.
These errors are not independent. An unclear objective changes which sources appear relevant. Weak context changes the plan. Missing tools prevent verification. Missing acceptance criteria allow all three to survive.
At this point, return to the carpenter's setup. The extraordinary table saw cannot reconcile two blueprints, reject rotten lumber, or decide the missing dimensions. Those controls belong to the workshop around it. The model is one station in the same kind of system.
This is why “I used the same prompt twice and got different answers” is not always the most interesting observation. The more important question is whether either run had a sufficiently constrained target and an evaluation capable of distinguishing success from plausibility.
Better models still need a designed workflow
Modern generative AI has steadily reduced the work required to express intent. GPT-3 could perform many tasks from text instructions and examples, but prompt phrasing mattered heavily. InstructGPT improved instruction following. Later systems added stronger reasoning, larger contexts, multimodality, search, code execution, tool calling, and routing between capabilities. The blade became sharper. The fence did not square itself.
Many organizations still paste an ambiguous paragraph into a universal chat box, request a production artifact, read the first screen, and publish. The model may be dramatically better, but the workflow still has one stage and no acceptance test.
The chat box also hides a toolchain. A coding agent can inspect a repository and run tests. A research system can open primary sources. A spreadsheet tool can preserve formulas and cell relationships. A text-only model without access to the workbook cannot produce the same auditable artifact, however fluent its explanation. Choosing the model, native tool, permissions, and deterministic checks is part of defining the work.
Using AI asks, “What can this model produce?” Designing an AI workflow asks, “What system will reliably produce, test, and improve the artifact we need?” Current guidance from both OpenAI and Anthropic separates models, tools, instructions, routing, evaluation, and human intervention rather than treating one prompt as the whole system.
This does not require elaborate autonomy. A good workflow might be a human with a checklist, a capable model, a browser, a test suite, and a clear handoff. The operator's responsibility is to design the path by which uncertainty is reduced.
CALIBER the workflow before blaming the blade
Now we can turn that systems view into an operating routine. CALIBER is the practical method.
| Step | Question | Working artifact |
|---|---|---|
| C, Contract the outcome | What decision, user, boundary, and success criterion define “done”? | Outcome contract |
| A, Assemble trusted context | Which sources are authoritative, current, relevant, and allowed? | Source set and conflict log |
| L, Load the right model and tools | What reasoning, modalities, software, and deterministic checks does the task require? | Routing plan |
| I, Isolate the stages | Which work should be researched, planned, executed, and reviewed separately? | Stage map and handoffs |
| B, Build checks before generation | What would prove the result wrong, incomplete, unsafe, or unfit? | Tests, rubric, and stop conditions |
| E, Evaluate against the contract | Does the artifact satisfy the original decision, not merely look polished? | Review record |
| R, Route failures and repeat | Which stage failed, who owns the repair, and what should the workflow learn? | Failure log and revised control |
The order matters. Teams often write checks after seeing the output, which lets the artifact redefine success. Build the checks first.
In the WebRTC example, contracting the outcome would replace “small” with a participant ceiling. Loading the right architecture would make topology a deliberate choice. Building the check first would require a scale test before the implementation could redefine success as “the demo connects.”
That is not bureaucracy for its own sake. On a small job, CALIBER might produce only a ticket, a source list, a test, and a named reviewer. The point is to spend judgment before polished output makes hidden assumptions expensive.
Anthropic's first-party analysis of roughly 400,000 Claude Code sessions found that users retained much of the planning work while the system handled more execution decisions, and that domain expertise correlated with success. The evidence is product-specific and observational, but it supports a useful operating intuition: delegation works better when the human can define and evaluate the work.
Professionals do not have magical prompts. They build reliable systems.
Sometimes the saw is defective
The strongest objection to this argument is correct: sometimes the saw is the problem.
Language models can invent sources, mishandle negation, violate constraints, expose bias, produce insecure code, flatten creative variety, and fail unpredictably at tasks that looked easy. Better workflow design does not repeal model limitations.
Nor should vendors escape responsibility. If an interface presents probabilistic output with the visual authority of a finished document, hides uncertainty, encourages one-click publication, or makes verification harder than generation, the product is helping create slop. Product defaults are workflow design at scale.
There is also a genuine capability frontier. In a preregistered experiment with 758 BCG consultants, participants using GPT-4 did better on work inside the model's evaluated frontier but were less likely to reach the correct answer on a selected task outside it. The published “jagged technological frontier” study warns against treating general fluency as uniform competence.
A 2024 meta-analysis of human and AI collaboration, covering 106 experiments and 370 effect sizes, found that human-AI systems beat humans alone on average, but did not beat the better of humans or AI alone on average. Outcomes varied with task type and relative capability. Collaboration is not automatically synergy.
These findings do not weaken the systems argument. They complete it.
The carpenter must inspect the actual saw as seriously as the setup. A bent arbor, weak motor, or unsafe guard remains a tool defect even in a disciplined workshop. The systems view assigns that defect correctly instead of using it to excuse rotten materials or missing measurements.
A workflow must include capability selection because models differ. It must include verification because models fail. It must include abstention and escalation because some tasks exceed the reliable frontier. It must include incentive design because cheap generation changes behavior. “The process matters” is not the same as “the model is never at fault.”
Responsibility remains distributed. Model builders own capability, calibration, safety, and truthful product boundaries. Product teams own defaults, provenance, and verification surfaces. Organizations own incentives and release controls. Operators own task definition, tool choice, review, and the decision to publish. No single layer can carry the others.
Better models do not eliminate specification
Model capability will keep improving. Better reasoning will resolve more implicit structure. Better agents will gather context, ask targeted questions, call specialist tools, test their work, and recover from failure.
Those improvements are valuable. They still cannot determine which business tradeoff you intended but never expressed.
An agent might notice that a board deck lacks a budget and ask for one. It cannot decide whether the organization should prioritize margin, market share, or regulatory caution without authority or evidence. A coding agent might infer the repository's test conventions. It cannot decide that a breaking API change is acceptable to customers.
Requirements engineering survives better intelligence because many requirements are not facts waiting to be discovered. They are commitments waiting to be made.
When the roof fails, can the team tell whether the blueprint, material, model, tool, execution step, or check introduced the fault? If not, iteration is only another guess. Expertise is recognizing which uncertainty matters, which evidence is sufficient, which tool fits, and which output is safe to release.
The durable advantage is process amplification. Strong systems make expertise repeatable. Weak systems make ambiguity scalable.
Inspect the workshop
Return to the doghouse. The carpenter still owns an extraordinary saw, and the setup is still part of the result. The blade, guard, and motor deserve inspection.
Then they square the fence.
They replace rotten material, reconcile the blueprint, write down the measurements, choose the right jig, and inspect each stage before the roof makes every earlier mistake expensive.
The same discipline applies to AI-assisted work. Pick one recurring artifact and write its outcome contract. Identify the trusted context, route the task to the right model and native tools, separate planning from execution, define the checks before generation, and record why failures happened.
Then run it again.
If the result improves, you did not prove that models are flawless. You proved that capability becomes useful through a system.
AI slop is real. So are hallucinations, homogenization, capability cliffs, and bad product incentives. But treating every broken artifact as a verdict on the model prevents us from seeing the larger machine that produced it.
Before you blame the saw, inspect the workshop.