B15 digest: 2024-04-16 to 2024-06-23 (16 posts)#

What this batch is#

Ten weeks in spring and early summer 2024, driven by several events: FHI’s closure, DeepMind’s Ethics of Advanced AI Assistants, GPT-4o, the Johansson voice row, and Safe Superintelligence Inc. Most of the posts are short, reactive and in his own prose. The exceptions are a student’s guest post (Vision Pro), a video-conversation intro (Levin, from the “Unplugged” series; there are no Modem Futura posts here), and long quoted passages from the DeepMind paper and the PCAST report.

Main ideas#

1. Safety is social: there is no absolute safety. The Sutskever post (2024-06-20) is the batch’s key statement of his risk thinking. He writes as a career risk professional: - “There is no such thing as absolute safety”. Harm is “a social construct, not a technological one”. - Risk, “the probability of harm occurring”, is the working form of safety. It is never zero, because zero risk “is only possible in the absence of change”. Acceptable risk is “governed by what people agree on”. - Complexity rises in steps: bridges, then chemicals and biological agents (dose-response, mechanism, acute versus chronic effects), then poorly understood emerging technologies, then superintelligence. - Harm is plural: to individuals (including dignity and autonomy), to societies (“spread of harmful ideas”), and to ecosystems.

This is his threat-to-value view of risk (B01–B06) aimed at AI-safety culture. The question SSI never asks is “who decides what “safe” means”.

2. Relational AI and “hyper-anthropomorphism.” Three posts form a sequence: - 2024-04-24 (assistants). Assistants and agents are the next phase: systems with “agency to change our lives — and even ourselves”. Neat ethical principles cannot contain them. - 2024-05-15 (GPT-4o). He coins hyper-anthropomorphism: “a concerted effort to create AI’s that are intentionally designed to engage our anthropomorphizing cognitive biases”. The mechanism is emotional trust through voice, “relational rather than transactional”. The stake is that we may “give away more of ourselves to them and their creators”. - 2024-05-21 (Her). He asks whether AI should be designed for romantic attachment at all.

This continues his older concern about AI exploiting cognitive vulnerabilities (the Ex Machina chapter, B08). The danger has moved into deliberate product design. The automated-social-science post (2024-04-21) adds a research route. Machines may learn about humans “faster and more effectively than we’re capable of learning about ourselves”, enabling “predicting and nudging human behavior”. He leaves the verdict open: “liberating, or deeply chilling”.

3. Tech leaders, sci-fi fantasy and responsibility theatre. This is the sharpest named criticism of OpenAI and Altman in the batches reviewed so far: - Altman “arrogantly plowed ahead” over Johansson’s refusal; - “disconnects between the talk around responsible innovation, and a reality that sometimes seems childish irresponsibility”; - “disdain for the dignity and rights of individuals”.

The 2024-05-26 post generalises: entrepreneurs “enamored with cool tech” miss the films’ social messages. The Ex Machina location essay (2024-06-16) revisits the film as a story of power and manipulation, adding reflections on natural versus artificial boundaries and on personhood “beyond human exclusivity”. The juxtaposition matters. In April he celebrates ASU’s OpenAI partnership and wants students “let loose” on ChatGPT. Five weeks later he condemns OpenAI’s conduct. He is both an enthusiastic user and partner and a critic of the company’s culture.

4. Existential risk: sceptical of the ideology, not dismissive of catastrophe. The FHI post (2024-04-28) sets out his physicist’s objection to Bostrom: - “philosophical elegance” overriding the second law of thermodynamics (the nanobot case); - Superintelligence favouring “philosophical speculation” over “practical reality”; - superintelligence, longtermism and EA spreading “like wildfire” into “tech bro” culture (Musk, Altman, SBF), with a “disdain for society”.

He still wants boundary-spanning futures thinking. It should be rooted in accountable public universities, grounded in humility, and treat future-building as “a collaborative effort”, not the work of an elite. The Baruchel post (2024-06-23) balances this. Eye-rolling at x-risk is “true at times”, but ignoring catastrophic technology risks “is in itself a risky strategy”. Low-probability, high-impact risks need context, not dismissal, and his nanotech history is the template: move from “gray goo” fantasy to real risks from engineered nanomaterials. In the Sutskever post he takes superintelligence as a premise and contests only how safety is framed.

5. AI in education and work: adopt experimentally, don’t shame. In the BlackBerry/iPhone post (2024-05-05) he warns against “early and naive adoption” and “tech lock-in”. Models are unstable across versions, and LLMs may not deliver what they promised or improve by scaling. His remedy is cautious experimentation, general-purpose tools over brittle enterprise tools, human teaching as the fallback, student freedom, and “investing in concepts, not products”. The ChatGPT-shaming post (2024-05-12) argues for norms agreed up front, humility, and AI “to support someone’s expertise, not as a substitute for it”. It uses a chemist’s line (“we should all be chemicals-free”) to make the point that AI is already everywhere.

Concepts appearing#

Risk and safety: no absolute safety; acceptable safety and acceptable risk; harm as a social construct; risk as the working form of safety; zero risk only without change; the engineering fallacy.
AI and relationships: hyper-anthropomorphism; emotional trust; relational versus transactional AI; the limits of principles-based ethics; machine prediction and nudging.
Leaders and futures: responsible-innovation “lip service”; cool tech versus social message; physical plausibility versus philosophical speculation; a knife-edge tipping point; collaborative future-building.
Adoption: low-probability, high-impact risks in context; naive adoption and lock-in; concepts, not products; ChatGPT shaming and the scale of comfort.
Being human: AI personhood and the natural/artificial boundary.

New or changed (relative to B01–B08)#

Safety as socially constructed and acceptability-based, stated in full and aimed at a frontier AI lab. Chemicals appear as a rung on a ladder of complexity, not as a template.
A new coinage, “hyper-anthropomorphism”. Manipulation is relocated into intentional design for attachment.
Surprise at pace. GPT-4o arrived “faster than I could have imagined”. He partly reverses his April view that the paper under-nuanced anthropomorphism.
Explicit wariness of historical analogies for AI. The calculator, internet, printing press and industrial revolution “all fail to capture the sheer uniqueness and profundity” of AI. This matters whenever his work is used as a lens on past-technology comparisons.
Scepticism about LLM scaling appears for the first time in the batches reviewed.
Stronger, named criticism of OpenAI and Altman, alongside continued institutional partnership.
A self-described move from nanotech to “a broader range of emerging technologies”.

Most important posts#

2024-06-20 ilya-sutskevers-safe-superintelligence-rethink: no absolute safety, acceptable risk, who decides.
2024-05-15 anthropomorphizing-gpt-4o: hyper-anthropomorphism and emotional trust.
2024-04-28 beyond-the-future-of-humanity-institute: Bostrom, x-risk ideology, and who should do futures thinking.
2024-05-05 blackberry-or-iphone-educational-ai: scepticism of analogies, adoption risk.
2024-05-21 openais-problem-with-the-movie-her (with 2024-05-26): AI leaders and sci-fi fantasy.
2024-04-24 navigating-ethics-of-advanced-ai-assistants: the agent frame and the limits of principles.