B08 digest: 2023-04-14 to 2023-05-12#

What this batch is#

This batch covers four weeks in spring 2023. ChatGPT was about five months old, and the Future of Life Institute “pause” letter had just come out. Maynard answers the moment largely by going back to his own earlier work and saying it still holds:

• He republishes the Ex Machina chapter of Films from the Future (2018).
• He refreshes a 2018 Risk Bites video on ten AI risks.
• He reuses a cut book draft on Pippard’s ladder.
• He revisits Bill Joy’s 2000 essay.
• He posts his draft explainer on Luddites.

Alongside these are hands-on ChatGPT posts (a prompt-engineering course, a dialect experiment). The mix is typical of him: enthusiastic about using the technology, and worried about how society is adopting it.

Main ideas#

1. Manipulation, not superintelligence, is the AI risk that matters most. This is the central, enduring claim in the batch. The republished 2018 chapter argues that the more plausible and more frightening risk is AI that learns human “biases, vulnerabilities, and blind spots” and uses them against us. He uses Plato’s Cave: humans build reality from shadows, and whoever controls the shadows controls us.

He is openly agnostic, even wry, about superintelligence. He recalls the 2017 Asilomar meeting (“a scientific meeting, not a religious convention”), calls Bostrom’s scenario “scientifically implausible” as of 2018, and prefers Russell’s context-bound idea of bounded optimality. Ten days later, the Bill Joy post brings this up to date for generative AI. It worries about “self-replicating” ideas, beliefs and ideologies, spread by machines “adroit at manipulating language” that can “slip under the checks and balances of our ability to reason and critique”. This is the clearest early bridge from his 2018 thinking on manipulation to his later concern with language and how people are formed by it.

2. Permissionless innovation, hubris and the lone innovator. The chapter sets out his critique of Adam Thierer’s “permissionless innovation”:

• Innovators act on what they think is responsible.
• A lone innovator cannot see the wider context, so other perspectives, “translators”, and “checks and balances around who gets to do what” are needed.

He makes the critique from the inside. He admits his own rule-breaking in the lab and his exhilaration at Musk’s Mars plans. He also admits a “glitch”: excessive caution stifles beneficial innovation. His position is to balance speed with forethought, not to oppose innovation.

3. Irreversibility, complexity and non-linear transitions. Two posts share a structural argument:

• The chapter traces a history in which harms became harder to reverse over time: pre-industrial, then industrial, then the nuclear and digital age plus globalisation. Consequences now “propagate through society faster than we can possibly contain them” (“a world made of kindling”).
• The Pippard’s ladder post adds tipping points and broken symmetries: in complex, tightly coupled systems “what has occurred in the past may not adequately predict what will happen in the future.”

This is his explicit rebuttal of the “past technologies turned out fine” argument. His view of climate is measured: a sudden global tip is unlikely, but complacency is unwarranted. He treats LLMs as possibly “a turn of Pippard’s ladder” while admitting “It’s hard to tell.”

4. How harm actually happens: eager adoption and diffused responsibility. The universities post gives a concrete account of how things go wrong. Transformative technologies “creep under our collective societal skin” through racing to adopt, the assumption that all will be well, and the belief that “any potential issues will be handled by someone else”. He adds a sharp aside about “who gets to write the history of technological successes”. He defends the pause-letter concerns as “well-founded”, not fear-mongering. He says universities, as agents of AI diffusion, must practise responsible innovation internally rather than make it “somebody else’s problem”.

5. Responsible innovation as the governance frame. The White House post:

• names the Stilgoe, Owen and Macnaghten framework (anticipation, reflexivity, inclusion, responsiveness);
• repeats his earlier argument, with Elizabeth Garbee, that it is “fiendishly hard to operationalize”;
• calls for transdisciplinary, anticipatory investment.

Its motto is: “if we wait until we have a problem with transformative technologies, we’ve probably waited too long.” He is relatively untroubled by industry here (“many AI companies are already investing heavily in responsible AI”). His worry is that government is late and reactive.

6. Reclaiming the Luddites: technology concerns are about values and power. Luddites were skilled technology users resisting industrialists who took away their livelihoods. Modern “Luddite” labels (for example ITIF’s award to Musk, Gates and Hawking) come from a techno-optimist, technology-is-neutral, “move fast and break things” worldview. Many objections (GMOs, for instance) are really about corporate power and social equity. He ends by claiming the Luddite spirit for responsible innovation.

7. Plausible over hyperbolic. He consistently separates “imaginable” from “plausible” futures. His ten-risk taxonomy runs from dependency, jobs, bias and opacity through misalignment and autonomous weapons to existential risk and “heuristic manipulation”: pluralistic, not ranked around x-risk.

Concepts appearing#

Permissionless innovation (critiqued); technologies of hubris; Plato’s Cave; imaginable versus plausible; bounded optimality; artificial manipulation / the “human club”; artificial emissaries; anticipatory and responsive governance; rising irreversibility; self-replication as metaphor; feedback loops between power and understanding; AI-bio-nano convergence; self-replicating ideas; tipping points and broken symmetry; advanced technology transitions; responsible innovation (AIRR); “somebody else’s problem”; Neo-Luddism; a ten-risk taxonomy.

Past technologies#

He uses past technologies mainly for structure, not literal comparison: nanotechnology (gray goo “deeply flawed”, yet like AI misalignment), GMOs, CRISPR and gain-of-function, the 1975 recombinant DNA Asilomar meeting, the nuclear age, climate change as the model for non-linear transitions, Industrial Revolution textile machines, and social media nudging as manipulation already happening.

What is new or shifted#

• Continuity is the main signal. He explicitly says the 2018 ideas are “more important today” and that the 2018 risk video is “not bad”.
• The language turn. He moves manipulation risk from the embodied, fictional Ava to language itself, and to ideas and beliefs replicating through generative AI. Manipulation could come from machines “or the people who make them”.
• Tone. He is more sympathetic to precautionary concerns (the pause letter is “well-founded”). He does not endorse superintelligence narratives, though he now allows that LLMs might lead toward AGI.
• Education. A new, practical strand: AI literacy for all majors, courses designed together with ChatGPT, universities as responsible adopters.
• His earlier AI claims are not revisited. The 2018 hardware-based claim that superintelligence is implausible is republished without comment.

Most important posts for understanding his thinking#

1. 2023-04-16 ai-and-the-art-of-manipulation. His fullest statement on manipulation risk, permissionless innovation, hubris, superintelligence skepticism and the imaginable/plausible distinction.
2. 2023-04-26 in-bill-joys-why-the-future-doesnt. Self-replication as metaphor; the feedback-loop gap between power and understanding; the language and belief manipulation extension.
3. 2023-05-04 tipping-points-and-broken-symmetries. Complexity, non-linearity, and why the past does not predict the future.
4. 2023-04-18 universities-need-to-be-investing. His account of how harms come through eager adoption and diffused responsibility.
5. 2023-05-12 unraveling-the-luddite-narrative. Technology critique as values and power; rejection of techno-optimist labelling.
6. 2023-05-05 us-white-house-embraces-responsible-innovation. The responsible innovation framework and anticipatory governance.