B24 digest: 2025-04-06 to 2025-04-08 (2 posts)#

What this batch is#

There are two posts, and only one of them counts.

2025-04-08 openai-and-studio-ghibli-style. An announcement of a Modem Futura episode on the GPT-4o “Studio Ghibli style” controversy. Skipped per user instruction.

2025-04-06 responsible-innovation-and-ai-acceleration. Only about 1,400 of this post’s ~20,500 words are his: the framing essay. The rest is a report written by OpenAI’s o1-pro on responsible innovation (RI) and responsible AI in the light of the AI 2027 scenario, with two annexes. He says he took part in the research and writing and edited the result, but it is AI-written. Its arguments are not evidence of his views: RI being “structurally outmatched”, IAEA- and CERN-for-AI, compute KYC, tripwires, moratoria, and comparisons with nuclear arms control, Asilomar 1975 and social media.

What counts is his framing. He calls the report “essential reading” and its first part “a very measured response”. He singles out Annex A, on RI in a US–China acceleration race, and adds in his own voice: “(not that well is the short answer)”.

Relevance: high. His own words are few, but they put a long-running core thread (responsible innovation) under pressure from AI acceleration.

Main ideas (his own prose)#

1. Responsible innovation may be futile against AI acceleration. Coming straight out of “a workshop on AI and responsible innovation”, his “first reaction” to AI 2027 was worry. Even as an edge case, it suggests “a near term future where current efforts to develop artificial intelligence responsibly seem futile.” The mechanism is a mismatch of timescales. Governance ideas and RI “depend on processes that are constrained by human timescales that are rather longer than those associated with intelligent machines”. He writes of “the futility of matching responsible innovation processes that can take years” to a race in which a month’s lag could mean “the difference between abject failure and world domination.” This is an unusually self-implicating worry for someone whose career is built partly around RI. It continues the unease seen in B15, where he wrote of “disconnects between the talk around responsible innovation, and a reality that sometimes seems childish irresponsibility”.

2. The deeper problem is cognitive and cultural: exponential-growth blindness. He says what worries him “just as much” is that “nothing about how we think, how we plan for the future, or how we develop approaches to ensuring better futures, is geared toward exponential advances that happen over months rather than years.” If AI 2027 happened, “we would most likely fail to recognize it — or would actively deny it — until it was too late,” because “we are really bad at wrapping our heads around rapid exponential growth.” He illustrates this with Al Bartlett’s 1978 bacteria-in-a-beaker thought experiment, which he met through the film Inferno: the beaker is half full at one minute to midnight. He asks: “what happens if we’re still planning for the world as it was at 11:00 PM when we get to the AI equivalent of 11:59 PM?” He adds a reflexive twist. Readers will treat this as “an intellectual exercise”, and that reaction is the point: “it always will feel like an intellectual exercise until it’s too late.”

3. Edge-case scenarios as stress tests, taken seriously but not believed. He calls AI 2027 “speculation — no more” and “highly speculative”. He lists four contestable assumptions: 1. AI companies building self-improving coding models to dominate the market; 2. employees reduced to “AI managers rather than AI developers”; 3. hardware and energy keeping pace; 4. a game-theoretic arms race that is “inevitable, no matter how bad an idea anyone thinks it is”.

Each “has its flaws” but is “not unreasonable as a starting point for imagining edge case scenarios”. Scenarios are useful “for exploring potential (if not necessarily likely) near term AI futures”. The practical upshot is precaution-like, though he never uses the word. Think through RI “just on the off chance that there’s a sliver of truth here”. He reports both the “too alarmist” pushback and the cautious welcome from “big names in cutting edge AI”.

4. Reasoning models as research partners. He describes “a mode of working that I’ve been finding increasingly useful recently”: using o1-pro “to develop nuanced and widely informed insights into complex questions”. He also has the model profile the AI 2027 authors’ “ideologies and perspectives” (Annex C) “so that o1-pro’s analysis can be contextualized”. So he attends to who is forecasting, but delegates that judgement too. Unlike the AI output he published in B17–B18, this is offered as serious policy analysis, not as an experimental curiosity.

What is missing from his own voice. He does not say what should replace or supplement RI, who should decide, or how serious existential risk is. He does not compare AI with past technologies. All of that sits in the o1-pro report, which he endorses only in general terms.

Concepts appearing#

Responsible innovation / responsible AI as slow, process-based governance.
Timescale mismatch: human and institutional time versus the time of “intelligent machines”.
Exponential-growth blindness and denial: Bartlett’s beaker, “one minute to midnight”.
Edge-case scenarios as planning tools, with contestable assumptions.
Tipping point.
The AI arms race as a game-theoretic trap.
Planning “on the off chance”.
AI-assisted deep-dive research (o1-pro) and attention to the forecasters’ ideologies.

What is new or changed#

Superintelligence and exponentials. In 2018 (B02, B08) he called himself “something of an agnostic” on superintelligence and attacked exponential extrapolation as a myth, on the grounds that every exponential stops. Future Rising (B17 excerpts) said the path to the future “is neither linear nor exponential”. Here he keeps the physical caveat: resource constraints “would slow or halt the exponential growth”. But he now uses the exponential frame the other way round. It is no longer a fallacy to debunk. It is a diagnosis of a blind spot in human cognition and in institutional planning. He is also willing to plan around a near-term superintelligence scenario as an edge case. The hedging stays heavy (“unlikely as I hope it is”). This is a move from skeptical agnosticism towards precautionary engagement, not a conversion.
Self-doubt about responsible innovation. He voices open doubt about whether RI, a framework he has championed for years, can work under AI-speed change, and he answers his own question about a US–China race: “not that well”.
Delegated synthesis. He uses a reasoning model for a policy-grade synthesis and calls the output “essential reading”. This is a step beyond his 2024 experiments with ChatGPT as author and moral reasoner.

Most important posts in this batch#

Only one post in this batch matters for understanding his thinking:

2025-04-06 responsible-innovation-and-ai-acceleration. His clearest statement so far of the worry that AI acceleration could make responsible innovation futile. He locates the problem in governance timescales and in human cognition (exponential-growth blindness and denial). It softens his earlier skepticism about fast AI takeoff. Only the ~1,400-word framing essay is his. The o1-pro report is material he curated and endorsed in general terms.

(2025-04-08 openai-and-studio-ghibli-style is a Modem Futura episode post and was skipped per user instruction.)