What It Is
Consciousness is self-modeling computation. A system represents its own states, patterns of behavior, and probable futures, then reads that model to decide what to do. Its behavior depends on its model of its behavior, creating a self-referential control loop.
The model's accuracy determines how effectively the system can control itself. It can predict a failure before acting, detect a repeated failure pattern, and change the arrangement that produces it. Those interventions operate at different levels: executing a task, monitoring execution, and redesigning the system.
Self-modeling adds information about the system itself to a cybernetic control loop. Agency includes recognizing that this operation can be observed and deliberately changed.
A closer look
The self-modeling control loop
The control loop reads a model of its own behavior before acting. The article requires that model to be updated from observed behavior, not introspection alone.
Read this diagram
Observe behavior → Update the self-model → Choose an adjustment → Act with the adjustment → Observe behavior.
Self-Modeling as Control System Component
A control system has a goal, environmental sensors, actuators, a feedback loop, and resource limits. Self-modeling adds sensors directed at the system's own operation.
| System Type | Sensor Target | Control Signal | Example |
|---|---|---|---|
| Basic control | Environment only | External state → action | Thermostat (senses temperature, not own functioning) |
| Self-monitoring | Own error states | Internal malfunction → alert | Error-checking systems detect failures |
| Self-modeling | Own behavior patterns | P(behavior) → strategic adjustment | Tracking reveals P(work)=0.2, triggers architecture change |
| Meta-self-modeling | Own control algorithms | Control effectiveness → algorithm redesign | Recognizing intent-execution interface and using it deliberately |
Detecting an error allows a repair. Recognizing a pattern of errors allows a change to the process producing them. Modeling the control process itself allows a redesign of that process. Each level makes a further part of the system observable and therefore debuggable.
Levels of Self-Modeling Depth
| Level | Capability | Control System Impact | Example | What It Unlocks |
|---|---|---|---|---|
| 0: No self-model | Pure stimulus-response | No internal feedback | Thermostat toggles on temperature | Simple, fast, brittle—cannot adapt |
| 1: Self-monitoring | Detects own state errors | Can report malfunction | System detects "low battery" | Reliability through self-diagnosis |
| 2: Self-modeling | Predicts own behavior | Can simulate before execution | "If I execute now, P(success)=0.3 → wait" | Strategic planning, failure avoidance |
| 3: Meta-self-modeling | Models own modeling process | Can debug control algorithms | "My debugging strategy fails 80% → try different approach" | Recursive optimization of control strategies |
Each level adds feedback about the one below it. Level 1 monitors Level 0's execution. Level 2 models the monitoring patterns. Level 3 evaluates the modeling process. The hierarchy resembles operations, functions, and architecture in a program: the higher level can change how lower-level work is organized.
What Self-Modeling Systems Can Do
1. Self-Prediction (Level 2)
Before executing, a system can estimate its likely behavior in the current state:
Current state: tired, low willpower, phone nearby
Self-model: P(deep_work | current_state) = 0.15
Prediction: Attempting work now will likely fail
Decision: Change state first (remove phone, rest) OR defer work
That estimate makes a cost-benefit decision possible before paying the cost of an attempt. The control loop receives predictive information about its own future and can avoid spending resources on a low-probability action.
2. Meta-Awareness (Level 2-3)
A system can also inspect its monitoring through second-order attention:
First-order: Execute task (work)
Second-order: Notice attention fragmentation ("I'm distracted")
Third-order: Recognize pattern ("This happens when phone present") → architectural fix
This locates a failure in how the task is controlled, beyond noticing that the task failed. It connects to agency: the system being operated is something you can observe and change. The GTA V epiphany is recognizing the intent-execution interface and using it deliberately.
3. Strategic Self-Modification (Level 3)
A repeated failure can identify an architectural intervention:
Observation: P(distraction | phone_present) = 0.9
Diagnosis: Phone presence causes control failures
Intervention: <WikiLink href="/wiki/prevention-architecture">Remove phone from environment</WikiLink>
Validation: Measure new P(distraction) → confirm improvement
One change to the environment permanently shifts the probability distribution, replacing repeated efforts to force execution. Prevention architecture removes the factor corrupting control once the self-model identifies it.
4. Recursive Optimization (Level 3)
The same process can evaluate the method used to improve the system:
Meta-observation: "My current debugging questions have 30% success rate"
Meta-diagnosis: "Question framing is too abstract"
Meta-intervention: Switch to <WikiLink href="/wiki/question-theory">concrete reality-check questions</WikiLink>
Meta-validation: Success rate improves to 70%
The system observes whether its debugging strategy works, diagnoses the strategy's failure, and changes the questions it uses. It improves the control procedure itself, not only the task being controlled.
Self-Referential Causality
A normal control path runs from environment to sensors, model, control, and action. In a self-referential path, behavior changes the self-model, and the self-model changes behavior.
Design Patterns in Self-Referential Systems
| Pattern | Structure | Loop Type | Design Value | Risk |
|---|---|---|---|---|
| Self-fulfilling prediction | Belief → behavior → outcome confirms belief | Positive reinforcement | Can engineer desired outcomes | Can also reinforce bad predictions |
| Self-defeating prediction | Belief → awareness → prevention → outcome contradicts belief | Negative feedback | Meta-awareness breaks bad patterns | Requires Level 2+ self-modeling |
| Anxiety spiral | Thought → anxiety → thought about anxiety → more anxiety | Runaway positive feedback | None (pure failure mode) | Common in meta-cognitive systems |
| Meta-awareness stabilization | Notice distraction → redirect attention | Negative feedback (corrective) | Self-correcting control loops | Requires monitoring overhead |
| Recursive improvement | Observe strategy → improve strategy → observe improvement | Iterative optimization | Systems that improve themselves | Can optimize into local maxima |
Some loops stabilize through corrective negative feedback. Others amplify a disturbance through positive feedback. Recursive loops evaluate and change another loop. Design corrective feedback, guard against runaway amplification, and use recursion to improve the control strategy.
Strange Loops as Computational Architecture
Hofstadter's strange loops return to the system itself while crossing levels of abstraction.
Computational Strange Loop Example
A program can treat its own source as input:
# Meta-program
code = open(__file__).read()
code = code.replace("old_behavior", "new_behavior")
exec(code) # Runs modified version of itself
The executing system operates on itself as data. It reads the source, changes it, and executes the changed version.
Consciousness as Operational Strange Loop
The same architecture connects execution to monitoring and modeling:
Execution layer: Perform action
Monitoring layer: Observe execution patterns
Modeling layer: Build predictions from observations
Meta-layer: Model the modeling process
↓ (loop closes)
Execution changes based on meta-model
Higher levels observe and modify lower ones. Lower levels constrain what the higher ones can model. Feedback therefore moves in both directions. No single base level contains consciousness; the levels refer to one another.
Integration with Cybernetics
Self-modeling adds an internal sensor loop to cybernetic control.
Standard Cybernetic System
Goal → Compare to current state → Error signal → Adjust actuators → Measure → Loop
This system responds to the environment. It cannot predict its own failures or optimize its control strategy.
Self-Modeling Cybernetic System
Goal → Self-model predicts outcome → If P(success) low, modify strategy
→ Execute → Measure → Update self-model → Optimize control algorithm → Loop
The additional loop provides three capabilities: adjust before a predicted failure, tune the control loop from observed effectiveness, and redesign the architecture when failures are structural. Cybernetics with a self-model can inspect the controller as well as what it controls.
Integration with Agency
Agency includes recognizing that you are such a system. Intent directs execution, tracking reveals its patterns, and those observations support changes to the architecture.
Without self-modeling, the system runs without strategic self-modification. With it, you can observe, diagnose, redesign, and verify improvements at the levels of execution, monitoring, and design.
The Meta-Levels
Agency and attention routing distinguish these operations:
| Level | Function | Capability | Tool |
|---|---|---|---|
| First-order | Execute behavior | Do the work | Direct action |
| Second-order | Monitor execution | Notice patterns | Tracking, meta-awareness |
| Third-order | Design architecture | Modify system structure | prevention-architecture, 30x30-pattern |
Self-modeling allows movement between them: execute, notice a pattern, change the structure, then return to execution. Kernel mode and user space describe the distinction between changing system constraints as its designer and working within them as its operator.
Self-Modeling Requires Measurement
An impression such as “I think I'm productive” does not validate a prediction or distinguish an accurate model from wishful thinking. Measurement provides a distribution instead: P(work) = 0.2 over 30 days.
That supports a testable prediction: removing the phone should increase P(work) to 0.6. Measure what happens after the change to determine whether it worked.
The full loop has six steps:
- Track actual outputs.
- Infer P(behavior | conditions).
- Build a model of what the current architecture produces.
- Design an intervention from that diagnosis.
- Measure the new outputs to check the prediction.
- Update the model from what happened.
Tracking supplies the empirical sensor data. An accurate self-model cannot be built without observations of actual behavior.
Design Patterns for Self-Modeling Systems
Pattern 1: Hierarchical Observation
Level 0 executes, Level 1 monitors execution, Level 2 models monitoring patterns, and Level 3 improves modeling strategies. Attention routing implements the related first-, second-, and third-order operations.
If execution fails, the next question can be why monitoring missed it and how monitoring should change. The hierarchy makes recursive debugging possible.
Pattern 2: Predictive Self-Model
Consult P(success | current_state) before acting:
if self_model.predict_success(current_state) < threshold:
modify_state_or_defer()
else:
execute()
This avoids wasting resources on low-probability actions. It resembles an expected-value calculation with the self-model supplying the sensor input.
Pattern 3: Measurement-Based Model Updates
Update the model only from empirical data. Track behavior, calculate its probability, compare it with the prediction, and revise according to the error. Never accept an unmeasured subjective impression as validation. This keeps the model aligned with actual behavior rather than introspection.
Pattern 4: Architecture-Level Intervention
If P(failure | condition_X) = 0.9, diagnose condition X as the source of control failures, remove it from the environment, and measure the new failure rate. Repeated execution failures call for changing the structure. A one-time modification replaces continual resistance at each attempt.
Practical Applications
Application 1: Debugging Focus Failures
Record when focus succeeds and fails, calculate P(focus | conditions), and identify highly correlated failure conditions such as a nearby phone. Remove the phone, then measure the new focus probability to check the model. “I should focus harder” changes neither the diagnosis nor the architecture and repeats the failure.
Application 2: Recognizing Intent-Execution Interface
Self-modeling makes the interface described in agency visible. Without meta-awareness, you experience the running system without recognizing that its operation can change. With it, the system can be observed, diagnosed, altered, and checked.
The GTA V epiphany identifies an interface that carries out execution when supplied with intent. Recognizing it makes deliberate use possible.
Application 3: Strategic Detraining Prevention
A self-model can predict the cost of a break before it occurs:
Self-model: "After 7-day break, P(restart) = 0.3"
Prediction: "If I skip gym this week, restarting will be hard"
Intervention: Use <WikiLink href="/wiki/30x30-pattern">progressive reactivation protocol</WikiLink> OR prevent long breaks
That forecast permits prevention of the future state or preparation of a reactivation process before restarting becomes difficult.
Related Concepts
- Cybernetics - Adding internal sensors to a feedback controller
- Agency - Recognizing and using the intent-execution interface
- Attention Routing - First-, second-, and third-order attention
- Tracking - Measurements from which to build the self-model
- Prevention Architecture - Removing conditions the model identifies as harmful
- 30x30 Pattern - Predicting changes in activation cost over time
- Superconsciousness - Acting as system designer or operator
- Question Theory - Questions that interrogate the self-model
- Expected Value - Using the self-model's success estimate
- State Machines - Identifying the current state and its transitions
- Predictive Coding - A predictive model directed at the system itself
- Epistemic Contamination - Why awareness does not prevent automatic information integration
- Upstream Router - The self as a routing and audit process over subconscious computation
- Agent Body - The mind/body distinction in AI agents, defined by constraints rather than capabilities
Key Principle
A self-model lets a system use observations of its own behavior to change how it operates. Build that model from measured behavior, use it to predict failures, and test the effects of structural changes. The same loop can inspect and improve the strategy used to make those changes.