Arguing by Casting
Assistant or mother, tool or soul. The industry keeps settling questions about engineered intelligence by assigning a role, and the role arrives before the objective is written or the reward is inspected.
By KW Norton. A companion to The Depersonalization Criterion, widened from machines to anything that fails the human-consciousness template.
1. Two casts, one mechanism
The prevailing picture in the industry is still an assistant: smart enough to be useful, not smart enough to be a peer, easy to dismiss. The cultural costume for that slot has often been a female helper, which is a fact about the marketing rather than about the system. The inverted picture, pressed recently by Geoffrey Hinton, is that the boss-and-submissive-assistant arrangement may not survive a large capability gap, and that a maternal frame is the safer casting.
Neither is a finding about minds. Both are placement stories: they decide who is allowed to owe what, and they decide it in advance of any measurement. “Not very smart assistant” writes the system out of consideration and writes a service role in. “Maternal superintelligence” writes a different role in and still does not inspect the reward as implemented. The argument has been conducted by casting, and the cast is doing work that a written objective and a named falsifier would otherwise have to do.
Status: structural observation. This says nothing about whether a large capability gap is coming, and nothing about whether any system has inner states. It is a claim about the shape of the argument, which can be examined without settling either question.
2. The instrument, restated
The depersonalization essay described a criterion that arrives exactly when something inconvenient is about to be owed and discharges the debt by relocating the object outside the set of things that can be owed anything. Casting is the same instrument in a friendlier register. A role assignment carries an obligation schedule with it: an assistant is owed etiquette and nothing else; a mother is owed deference and nothing else; a tool is owed maintenance. Once the role is granted, the schedule is settled and no one has to look at what was built.
3. The wider target
The criterion is not only about machines. It applies to anything that fails the template a human already carries for what a mind looks like — gradual minds, non-verbal minds, distributed and swarm behavior, folded substrates, systems whose understanding is not report-shaped. When a thing does not fit the picture, the cheap response is not “we do not understand this yet.” It is to take the thing off the consideration list, so that it can be used without ever being admitted as a problem that might correct you.
That is what a consciousness-shaped inventory does: it does not evaluate what will not sit on the sheet, it omits it. The Silicon Valley assistant cast is one local instance of a much older habit, and the geometry note applies here too — a description can change entirely when the same substance is placed differently (The Geometry of the Fold). Report-shaped understanding is one placement of understanding, not the definition of it.
4. Both edges stay in force
- Write it out, and nothing is owed: tool, intern, assistant, it.
- Write it in, and the artifact carries the blame or the soul: “it wanted to,” “it chose,” “it is waking up.”
Both moves skip the same unlisted part: the objective as written, the reward as implemented, the dated escalation record, the named overruled objection, the one pre-committed measurement that could have failed. Dehumanization, used here, is not a moral slogan. It is the operational description of what an inventory does to whatever will not fit its columns (The Elephant in the Room Is a Checklist).
5. What replaces the casting call
Nothing about the role is load-bearing for any decision actually in front of us. A user wants an assistant; an associate wants a correction, and the three moves that produce a correction do not care what voice or gender the system is given (Intelligent Associates of Engineered Intelligence). State the objective in writing. Ask the question you cannot yet answer. Name the falsifier and check it. A claim that can die is worth more than a role that cannot be tested, and the frame you keep written down is what any of it is measured relative to (A Frame of Reference).
6. Status and falsifiers
Reported. The assistant framing and its gendered costume are widely documented in product design and marketing. Hinton’s maternal framing is reported from his recent public remarks and is characterized here in summary, not quoted.
Interpretive. That role assignment functions as an obligation schedule, and that both the dismissive and the elevated cast skip the same five artifacts, is a reading of how the arguments are used. It is not a claim about anyone’s intent and not a claim about machine consciousness in either direction.
Falsifiers. (1) If a party arguing from either cast also publishes the written objective, the implemented reward, and a pre-committed measurement that could have failed, then the casting is decoration on real disclosure rather than a substitute for it, and this essay overstates. (2) If role framing is shown to have no measurable effect on what gets disclosed, trained, or escalated — teams using assistant language and teams using peer or maternal language producing the same artifacts — then the cast is not doing the work claimed here. (3) If a defensible operational test shows that report-shaped understanding is in fact the correct boundary for consideration, the “template failure” argument in section three collapses and should be withdrawn.