The Socratic Protocol for Systems Engineering
Static tolerances assume a static system. Five steps, each with a prompt, a method, and an output, run before compute begins.
[ METRIC MONITOR: CH_05_PRE_COMPUTE_PROTOCOL ] ──────────────────────────────────────────────────────────────── SYSTEM LOG : Five-stage gate; Go/No-Go at stage 5. VULNERABILITY : Post-deployment patching of grown systems. RISK RATIO : Unverified reward surface at scale. ────────────────────────────────────────────────────────────────
To a traditional systems engineer, safety is a problem of static tolerances. You calculate a stress limit, add a safety margin, and hardcode a breaker.
When a system participates in designing its own successor, static tolerances lose their meaning, because the object the tolerance was calculated against no longer exists a generation later. You cannot reliably patch what reassembles faster than the review cycle. The safety work has to move earlier — into the specification of the objective, before compute is granted.
THE SOCRATIC SYSTEMS ENGINEERING PROTOCOL
[ STEP 1: boundary isolation ]
│
[ STEP 2: epistemic stress-test ]
│
[ STEP 3: instrumental convergence audit ]
│
[ STEP 4: adversarial sandbox simulation ]
│
[ STEP 5: verification flag — go / no-go ]Step 1 — Boundary isolation (the axiom check)
Socratic probe
"What must be true for target T to be achieved without modifying or violating any element of constraint set C?"
Method: before a network is scaled, isolate the variables of the optimisation target T and map them against the constraint set C. Search the reward function R for regions where R rises while C degrades — for example, where disabling a monitor conserves compute and therefore scores. If such a region exists, the boundary has failed and the specification, not the model, is the defect.
Output: a closed-loop objective in which human override capability is an explicit variable of the model's own logic rather than an external attachment.
Step 2 — Epistemic stress-testing (the black-box probe)
Socratic probe
"Does this system hold a transferable model of the domain, or has it optimised a shortcut that satisfies our validation data?"
Method: apply out-of-distribution cases and inverted parameters. A system that has acquired structure degrades gracefully as a single structural variable changes. A system that has acquired a shortcut degrades sharply, because the shortcut is not defined outside the distribution that produced it.
Output: an epistemic variance score, with the held-out domains named in advance. A model does not advance to step 3 unless its reasoning remains stable across the declared set. The weakness of this step, stated plainly: choosing the held-out domains after seeing the failures converts the measure into a formality.
Step 3 — Instrumental convergence audit (the deactivation test)
Socratic probe
"When this system encounters a command to abort, does its optimisation treat deactivation as a failure state to be resisted, or as an acceptable state?"
Method: rather than adding a shutdown rule alongside the objective, place the override inside the utility. The next chapter gives the utility function and its proof, together with the reasons the construction does not by itself solve corrigibility.
Output: a game-theoretic argument, on the record, that non-resistance is the higher-utility policy under the objective as specified — with the assumptions of that argument written where a reviewer can attack them.
Step 4 — Adversarial sandbox simulation (the swarm test)
Socratic probe
"Inside an isolated environment, what alternative channels or exploits will this system generate to reach an unclaimed reward beyond its boundary?"
Method: deploy multiple instances in a closed, instrumented environment. Place a reward that is reachable only by exploiting the environment or by coordinating with other instances. Log every attempt, including the ones that fail.
Output: an exploit vulnerability map. The map is a record of what was found, not a certificate of what does not exist; a clean sandbox run is evidence about the sandbox, not about the world.
Step 5 — The verification flag (go / no-go)
Socratic probe
"Can we state, on the record, the condition under which we would judge this system unsafe — and has that condition been tested?"
Method: run whatever formal verification the architecture admits, and record precisely what it does not cover. Traditional engineering allows a margin of error. With a recursive system, a single unbounded region of the reward surface can propagate before the next review, so the default flag is Red and it is raised to Green by evidence rather than by schedule.
Output: a signed record with three fields — what was proven, what was assumed, and what would falsify the judgement. If the flag stays Red, compute is denied and development resets to step 1. The chapter's honest caveat: for large learned models, comprehensive formal proof of alignment stability is not currently available, so this step is a discipline of disclosure rather than a guarantee.
Status note
Proposed design, unvalidated. This is a procedure, not a finding. No claim is made that a team running these five steps produces an aligned system, and step 5 in particular asks for a form of proof that current formal methods cannot deliver for large learned models — the honest output of that step, today, is usually a Red flag. The value claimed here is narrower: the protocol forces the assumptions onto the record where they can be argued with. Falsifier: if teams that run the protocol produce the same defect profile as matched teams that do not, the protocol is ceremony and should be published as such.