Days of Future Past: What Sort of AI are We Building?

I welcome the recognition that advances in engineered intelligence are outpacing our understanding. I also welcome the insistence that the questions raised by this technology cannot be left to technologists alone. The achievements are extraordinary. My concern is not that engineers have contributed too much, but that the picture through which we evaluate their contribution may still be too narrow.

The proposed standards body addresses important questions: dangerous capabilities, security, independent testing, and evaluations that change as models change. These deserve serious attention. But the distinction between technical foundations and the “further” economic and philosophical questions worries me. Even if that is an order of exposition rather than a proposed timetable, it risks making the human questions look like matters to be addressed after the machinery has been secured.

They are already inside the machinery. Who chooses what counts as capability? Which forms of success become benchmarks? What does a model learn to reward, and what do people learn to accept from it? Who can challenge an evaluation that declares a system ready for use? These decisions shape development itself, not merely society’s later response to a finished instrument.

It is relationships all the way down. An answer does not enter an empty room. It reaches a person with a particular education, purpose, vulnerability, and capacity to question it. That encounter occurs within institutions that distribute authority and consequences. A fluent answer may enlarge someone’s understanding; it may also make it easier to surrender judgment. The relevant question is not only whether the model performed correctly, but what the exchange enabled the human to understand and decide.

This is where I would place Socratic education: not as a universal cure or another prescribed curriculum, but as a possible way to strengthen the human half of the exchange. Can people question the answer, the framing of the question, and the guide offering both? Can the guide itself be changed by what is learned? An educated public is not simply a public that has been informed about the technology. It must have opportunities to practice thinking with it without handing over the verdict.

The proposed funding and governance arrangements deserve the same questioning. Industry resources and technical expertise would plainly be valuable. But a body funded largely by the industry it assesses needs more than an assurance of independence. How would conflicts be handled? What standing would affected people have to challenge its judgments? What prevents “Frontier Lab” prestige from becoming a substitute for demonstrated responsibility? These are design questions, not allegations about the people proposing the framework.

I share the excitement about new materials, energy research, and discoveries that this instrument may help make possible. But abundance is not yet an answer to questions of distribution, access, or judgment. Nor does a prediction about the arrival of general intelligence establish that the categories used to recognize it are adequate.

The world is changing, and so are the people constructing and encountering these systems. We need standards capable of following that transformation without mistaking what they can measure for the whole of what matters.

The future is not yet written. That is precisely why the wider questions belong at the beginning: what are we constructing, which relationships will sustain it, and who can require us to reconsider?