Intelligence Design

A Note on This Article

These are speculative ideas about agent architecture, developed while building agent systems. The heuristics and mental models remain works in progress requiring empirical validation. They provide a way to reason about design, rather than proven laws.

The article distinguishes discardable candidate errors from routing errors that affect later stages.

Read this diagram

Generate candidates → Evaluate them → Select an acceptable result → Pass it onward.

The Wrong Mental Model

Suppose a generator produces ten candidate answers and one is bad. A later evaluator can discard it. If a router sends a request to the wrong specialist, however, every later step can perform the wrong task correctly. Improving either component by tweaking its prompt misses the difference between their roles in the system.

The familiar input → function → output model encourages treating an agent as a deterministic executor: give the instruction, obtain the result, and revise the instruction if the result is wrong. An LLM instead samples a probability distribution over outputs. The same prompt can produce different results. Hallucination, drift and misunderstanding are properties of this substrate that the architecture must accommodate.

The design task is to obtain reliable outcomes from unreliable components. That requires control over how calls are arranged and assessed, as well as what each prompt says.

The Universal Pattern

Intelligence reliably produces good outcomes under uncertainty. Its common operation is to generate possibilities and then filter for good ones. Every intelligent system runs a version of that loop:

Evolution obtains organisms through variation and selection. Science tests hypotheses instead of requiring a correct theory on the first attempt. Markets fund ventures and use profit and loss to select among them. The material differs across these systems, but repeated generation and filtering produce the useful result.

Agency in System Design

Agency means producing effects rather than only responding to events. An agent designer can respond to each failure with another prompt revision, leaving reliability dependent on the next sample. Or the designer can arrange a pipeline that detects, rejects and learns from failed samples.

The second approach changes the probability of a correct result across the system. Probability Space Bending describes this intervention in a distribution: build conditions that move probability toward the desired outcome instead of trying to predict which individual attempt will succeed.

The Signal Metaphor

A signal passing through a noisy channel has to be reconstructed at the receiving end. In an LLM call, the intent is the signal and the model is the channel. Several errors can corrupt it:

• Hallucination adds false content.
• Drift loses the thread during a long context.
• Misinterpretation substitutes a different meaning for the intended one.
• A format error violates the expected structure.
• A knowledge gap leaves the model without required information.

The result contains signal and noise. A system can improve their ratio across several stages even when no individual call becomes completely reliable.

Why Single Attempts Fail

A detection threshold is the minimum signal strength distinguishable from background noise. One LLM call supplies one sample, regardless of the effort spent crafting it. Trying to make that single attempt exceptional is an intensity strategy. It fails when unusually good prompts cannot be produced reliably, when model variance creates a high noise floor, or when a nonzero hallucination rate leaves attempts below the threshold.

The reliability comparison assigns these outcomes to a component with P(correct) = 0.7:

Prompting improves the component's 0.7 probability. The architectural intervention combines components to obtain the higher system-level probability. Independence and correlated error matter to this comparison, as discussed under amplification strategies.

The Core Primitive: Amplification

Repeated attempts make a low-probability success more likely to appear somewhere in the set. If each attempt has P(success) = 0.02:

The probability compounds rather than increasing linearly. The system must then recognize the successful attempt:

Generate N → Auto-evaluate → Select best

Several techniques arrange this operation differently:

The shared mechanism is generating alternatives and applying selection, although the candidates and selection procedure differ.

The Structural Requirement

Filtering provides leverage only when evaluation is cheaper and more reliable than generation. Deterministic tests or schemas make it work perfectly. An LLM evaluator can work with some noise when checking the answer is easier than creating it. If evaluation is just as hard as generation, the extra call doubles computation without resolving the original problem.

Other tasks lack that asymmetry. Creative writing has subjective judgments and no ground truth. Open-ended reasoning can take as much work to validate as to perform. A novel problem has no known answer to compare against, and taste can make the scoring function as uncertain as the generator. These differences give code generation much more filtering leverage than creative writing.

Distribution Control Variables

The distribution changes through concrete design choices:

A pipeline combines these controls. More samples are useful only if the selected result is better; a narrower task is useful only if its output can be incorporated into the larger one.

Amplification Strategies

The five-call, 0.97 comparison assumes independent calls. Calls to the same model with the same prompt instead share systematic biases, so repeating them does not supply independent evidence.

Diversity addresses the shared error. Self-consistency changes the reasoning path rather than merely rerunning an identical route. AlphaCode generates structurally different programs. When errors are correlated, sampling different regions is more useful than taking more samples from the same one.

Signal Function Taxonomy

A call's role determines where failure propagates and how much reliability it needs.

Source Functions (Generate Signal)

A generator produces candidates, drafts or options. Individual reliability can be low because downstream filtering removes bad outputs: twenty proposed approaches might yield three worth pursuing. A planner decomposes an intention into executable steps. Its reliability must be medium-high because later work depends on that structure; the plan needs validation and room for revision before execution.

Routing Functions (Direct Signal)

A router or classifier chooses the next path. Incorrect routing can corrupt all later work, so reliability must be very high. Constrained outputs, explicit categories and fallbacks limit that risk. An orchestrator coordinates execution across agents and needs the same reliability because it controls the whole flow. Simple logic and deterministic operations reduce the amount entrusted to an LLM.

Transformation Functions (Modify Signal)

A specialist performs one bounded transformation. Medium reliability is acceptable when attempts can be retried and filtered; scope should be clear. A translator changes representations, such as natural language to SQL or prose to structured data. A compressor summarizes or distills while preserving the essential information. An extractor isolates entities or other requested information from noisy input. A synthesizer combines several sources into one coherent result.

Filtering Functions (Reduce Noise)

A validator or evaluator checks criteria and supplies a direction for correction. Its reliability must be high because bad feedback produces bad learning. Multiple validators, explicit rubrics and cross-checks support it. A critic reviews generated content for errors before use. A recovery function classifies failures, adjusts parameters and chooses a fallback.

Memory Functions (Persist Signal)

Memory stores and retrieves information across sessions. Corruption persists and propagates, making reliability important. Structured storage and validation at the point of writing protect later uses of the record.

Composition Patterns

Pattern 1: Reliable Output from Unreliable Source

Generator(n=10) → Evaluator → Filter(threshold) → Output

When verification is cheap, ten noisy candidates can be evaluated and filtered to produce one reliable answer. The generator does not need to avoid every mistake because the later stages can recover from them.

Pattern 2: Domain-Appropriate Processing

Router → Specialist[domain] → Validator → Output

Different inputs require different specialists. The router chooses the domain, the specialist performs the work and the validator checks it. An incorrect route can make every subsequent stage solve the wrong problem, so reliability is concentrated at that first choice.

Pattern 3: Iterative Refinement

Generator → Critic → Refiner → Critic → ... → Output

This arrangement helps when a reliable critic can identify a useful change. Each pass removes noise or adds signal, and the revised output returns for another assessment.

Pattern 4: Parallel Decomposition

Planner → [Specialist × N in parallel] → Synthesizer → Output

Independent subtasks can run at the same time. The planner defines the separation, specialists perform the pieces, and the synthesizer recombines them. The independence supplies the opportunity for large-scale parallelism.

Pattern 5: Generate-Test-Refine Loop

Generator(n) → Tester → [passing] → Select best
                     → [failing] → Analyzer → Generator(n, with feedback)

Tests separate passing outputs from failures. Failure analysis supplies feedback to the next generation round rather than ending the attempt. This is the generate-test-refine arrangement used by AlphaCode.

Foundational Observations

Prompting Is Necessary But Not Sufficient

Each node still needs a good prompt: it shifts the distribution toward the desired result. Single-call improvement has a ceiling, however. Volume, filtering and decomposition can take the system beyond that ceiling. Prompting designs the component; amplification and filtering determine how the components work together.

Reliability Requirements Vary by Function

A router's wrong decision corrupts downstream work, while one bad generator output among ten can simply be discarded. Where reliability is critical, deterministic fallbacks, constraints, repeated verification and limited LLM dependence are appropriate. Where filtering can recover, more freedom and greater candidate volume can be useful.

Noise Budget Is Finite

Every stage can add error. If a router directs three agents to the wrong task, all three efforts are wasted. If nineteen of twenty generated candidates fail a filter, the twentieth can still make the system succeed. The noise budget belongs where failure is recoverable; routing and orchestration need to minimize it because their errors cascade.

Evals Measure Distributions

One successful test does not establish production reliability, and one failure does not establish universal failure. A prompt change can also appear helpful or harmful depending on which sample is observed. An evaluation uses repeated runs—100 in this example—to obtain P(correct), then measures whether architectural changes move that probability past the required threshold.

The Framework Applies Where Asymmetry Is Largest

Code, structured extraction, factual work and format compliance have tests, sources or schemas. These are also the domains where most production agent use cases live. Their available checks give generation and filtering its largest advantage.

Evidence

Published results support the generation-and-filtering pattern:

These methods use volume and filtering instead of relying on a better single-shot prompt. They do not establish the proposed signal-function taxonomy, its reliability requirements, the best compositions for each domain or generalization to every agent task. Those claims still require empirical measurement.

The Meta-Principle

The same intervention appears in agency and forcing functions. Requiring willpower for each gym visit acts on individual instances. Changing the surrounding architecture changes P(gym) across future instances. Agent architecture similarly changes P(correct) across outputs.

The engineering is identical across these substrates: the intervention changes the generator of outcomes rather than one outcome. Level 4 agency calls this engineering the probability distribution. A weather forecaster describes a distribution while a climate engineer changes it; an intelligence designer changes what an agent is likely to produce.

Related Concepts

• Agency concerns causing effects through system design.
• Probability Space Bending changes distributions of outcomes.
• Ladder of Agency places distribution engineering at Level 4.
• Forcing Functions changes probability through structure.
• Prevention Architecture makes failure paths unavailable.
• Signal Boosting amplifies weak signals through volume and filtering.
• Effective AI Usage develops practical AI workflows.
• AI as Accelerator examines the collapse of implementation complexity.
• Cybernetics supplies feedback and control.
• Statistical Mechanics reasons about distributions and microstates.