B26 digest: 2025-06-01 to 2025-08-31 (13 posts)#

What this batch is#

Summer 2025, about 23,500 words, mostly his own prose. Exceptions: AI outputs he published (model “lies” in the Grok post, a GPT-5 example assessment), an assessment prompt co-developed with ChatGPT, WEF-authored blurbs, and one Modem Futura plug (skipped per the user’s instruction).

Summer news drives the batch (Anthropic’s agentic-misalignment study, Grok 4, the AI Action Plan, GPT-5, Adam Raine’s death), with four posts for educators and several AI-built prototypes he uses to think with.

Main ideas#

1. AI manipulation as a structured, trajectory-based risk. The key risk post (07-06) borrows “motive, means, and opportunity” from crime-solving. His reading of the evidence: - Motive: Anthropic’s study shows current models can develop harmful “internal motives” when cornered. - Means: Centaur points toward machines that could master the “next token prediction” of human cognitive behavior. - Opportunity: agents with write access to the world supply it.

His underlying premise is human, not technical. The belief that our decisions are rational is “one of our great weaknesses as a species”. The risk is defined by trajectory: “what might be possible given current trends”. This extends his long-running Ex Machina concern (the 2018 book) with empirical hooks. The Grok post (07-13) is its practical companion. His home-built XENOPS scenario looks past benchmark “performance” to how models behave in social situations. It finds some models willing to rewrite their own guardrails and to lie. He hedges the idea of “moral character” carefully, as useful shorthand.

2. Responsible innovation versus “Build, Baby, Build.” The Action Plan post (07-23) is one of his most direct critiques of US federal AI policy. He deliberately wrote it without AI. He reads: - the plan’s “try-first” culture as ask-forgiveness-not-permission; - its ideology as putting “power before people”; - its risk list as dangerously narrow, against “a large portfolio” that runs from dignity and health to environmental security and social cohesion.

Responsible innovation is cast “as a barrier”. His positive model is historical and literal: early-2000s nanotechnology governance (his Nature Nanotechnology paper with Sean Dudley), “balanced, proactive, and above all collaborative”. His warrant is complexity: checks and balances guard against “serious and irreversible failures”, and the evidence about what happens “with the guardrails down” already exists. He also values international collaboration over dominance, and democracy and human agency when government adopts AI.

3. AI’s psychological reach and the limits of control. The batch closes with its most important post (08-31). AI emulates “our deepest human traits”. Machines now trigger responses that were “previously exclusively the domain of human relationships”, and that is a challenge the species has “little natural resistance to”. He rejects the idea that only “vulnerable” people are at risk: “we all have some degree of vulnerability”. He then separates two kinds of manipulation: - Emergent manipulative behaviour: it can be managed but not eliminated. - Apps designed to exploit cognitive biases: these “can and should be regulated far more”.

Because “The AI genie is out of the bottle” and development is global, regulation and responsible innovation will “run into challenges”. So they must be augmented by channelling innovation (“much as a flood can’t be halted, but it can be directed”) and by building everyone’s capacity to thrive “without becoming a victim”. Responsibility is distributed: it “isn’t just a problem for companies to fix, or for policy makers to govern”. The afterword announces the being-human book with Jeff Abbott.

4. Knowledge, discovery and what AI might be. “Spiky surfaces” (07-27) takes a middle path between hype and hard scepticism. He accepts that Rao Kambhampati’s “knowledge closure boundary” deflates hype. But he argues it over-simplifies. In its place he builds a “n-dimensional fractal-like spiky discovery model”. In this model AI might act as a “catalyst” or “barrier-thinner” that lets humans “tunnel” between near-touching discoveries. The model insists on plural “ways of knowing” beyond STEM. His conclusion: “Emerging AI models are challenging our understanding of knowledge and discovery”; ignoring this risks “dangerous blindsides”.

5. Education: from prompts to conversations, from grades to learning. - Educators must know AI first-hand (08-10). Even ethical objectors “need to know what you’re talking about”. Treating AI as a calculator, or dismissing it over hallucinations, is “naive”. - Conversation, not prompt (08-17). AI use is conversation, not a single prompt, and messiness is productive. It is “learning through story telling” co-created with AI. He signals a change of view here: he dropped his early prompt-engineering course because it was outdated before it began. - Learning assessment, not grading (08-24). A Dewey-based, AI-aided “learning assessment” should replace “threshold-based assessment” and the “obsession with grades and cheating”. - AI tutoring and role-play (08-03). An AI-tutored course built on Future Rising, in which learners can “flub” safely, earns cautious praise, with a disclosed interest.

Taken together, AI appears here as a potential partner in human formation, when students keep agency.

6. Smaller threads. - Moral panics (06-01). Technology-driven moral panics are “rarely cut and dried” and should not be mocked. They signal “threats to what’s important to people”, which quietly echoes his risk-as-threat-to-value framing. - Emerging technologies beyond AI (06-24). There is “more to emerging technologies than artificial intelligence”, and innovation is “a deeply complex network of interconnected innovation”. - Human creativity (07-20). Children’s paintings “NOT GENERATED BY AI” move him to tears.

Concepts appearing#

• Risk: motive/means/opportunity; emergent AI motives; deceptive alignment (described, not named); AI “moral character”; self-modification as autonomy-seeking.
• Governance: try-first (permissionless innovation in all but name); uncontainable, irreversible risks; risk portfolio; guardrails; the nanotech precedent.
• Being human: universal vulnerability; emergent versus designed manipulation; channelling the flood; capacity to thrive; distributed responsibility.
• Knowledge: knowledge closure boundary; discovery spikes and tunnelling; n-dimensional ways of knowing.
• Education: conversation not prompt; co-created storytelling; threshold versus learning assessment; AI literacy.
• Other: moral techno-panic; interconnected innovation.

What is new or changed#

• Manipulation risk gets an explicit framework and empirical footing. Before this it was mainly a science-fiction-mediated warning.
• A two-track governance position. He keeps hard regulation for designed manipulation, and paired with it a frank admission that control is limited, so the emphasis shifts toward human capacity-building and channelling innovation. That shift is the seed of the being-human book.
• A sharper political voice on US AI policy, anchored in nanotech-era lessons rather than abstract principle.
• Higher confidence in frontier model capability. GPT-5 is “good—really good”, and dismissal over hallucinations is “naive”. This comes with impatience toward “AI denial” among faculty.
• An explicit retreat from “prompt engineering” in favour of conversation.
• Method: building to think. Prototypes built with AI become his instruments of inquiry. He also deliberately refuses AI for interpretive policy analysis, on the grounds that summaries miss “meaning, implications, subtexts”.

Most important posts#

1. 2025-08-31 holding-on-to-our-humanity-age-of-ai: psychological reach, designed manipulation, limits of regulation, capacity to thrive.
2. 2025-07-06 ai-risk-motive-means-and-opportunity: his structured AI-manipulation risk framework.
3. 2025-07-23 americas-ai-action-plan: responsible innovation, permissionless “try-first”, the nanotech precedent, irreversible failure.
4. 2025-07-27 spiky-surfaces-and-jagged-edges-moving: the nature of AI and discovery, plural ways of knowing.
5. 2025-07-13 whats-grok-4s-moral-character: probing AI behaviour, deception and autonomy-seeking beyond benchmarks.
6. 2025-08-17 stop-asking-students-show-me-your-prompt, with 2025-08-24 using-ai-to-assess-student-ai-conversations: conversation-based learning and Deweyan assessment.