Late Lessons, Jensen Huang and AI
Markdown version

B08 perspective notes: 2023-04-14 to 2023-05-12 (12 posts)#

These notes read the batch for how Maynard thinks, not for the concepts he names. All twelve posts were read in full. Quotes are exact, including his typos.

Evidence rules applied

Context. The batch falls five months after ChatGPT’s launch and weeks after the FLI “pause” letter. Much of it brings back older work: a 2018 book chapter, a 2018 Risk Bites video, and a cut book draft on Pippard’s ladder. He sets this beside new experiments: a ChatGPT course, dialect play, pizza meetups. The batch shows him checking his own earlier thinking against a moment he calls “an AI lifetime” later (2023-04-24 ai-risks-primer).

1. How he thinks here#

He opens from a concrete encounter, not a thesis#

Most posts start with something small that happened to him:

The big claims come later and grow out of the particular. The Pippard post moves from a rope ladder to climate to LLMs to a choice about how society proceeds.

He finds out by doing, and shows the working#

The tardigrade post is the clearest record of his method as a sequence of moves:

  1. He suspects the story is “manufactured”.
  2. He tries and fails to find the source paper, then asks his grad student to search.
  3. He uses ChatGPT to write a fake version of the story “just to see how close it came”. It “came scarily close!”
  4. He holds a nagging counter-signal: “the science here makes sense” (the Dsup protein is real and plausible).
  5. He tells his student it “looks like this is a made up story”.
  6. He reverses when Jieshu “came up trumps” and finds the paper.
  7. He reads it himself and finds the “super soldiers” framing is not in it.
  8. He adds a public update after a biologist on Twitter points out that Dsup increases DNA damage in neurons: “if something looks too good to be true, it probably is … especially in biology!”

Two kinds of evidence (where the story came from, and whether the science is plausible) are weighed against each other and left open until the primary source settles it. Corrections are welcomed rather than hidden.

The same experimental stance runs through his teaching and community work:

Play as a serious instrument#

In 2023-04-24 gettin-ai-reet-in-tschools-nas-ttime he was “playing around with ChatGPT’s mastery of dialect (for serious reasons)”. He enjoys the result, calling its phrases “quite delicious” (he is “a Lancashire lad”). The play still produces a real question: “how, in the long run, this will impact effective communication as well as the preservation of local dialects—and their associated cultures.”

Play is how a new risk and value question comes into view. He is playful even with ideas he takes seriously, admitting he is “being a little playful with Bostrom’s ideas” (FFTF p.169).

Analogy by structure, not surface#

He pulls out the mechanism and drops the literal content:

Reframing the question to get at what matters#

He sidesteps definitional debates:

Plausibility tests, grounded in physical science#

These are a physicist’s instincts. They are always paired with an admission of fallibility (see section 3).

Holding tensions openly#

He names the “glitch” in his own argument against permissionless innovation: without gung-ho innovators, “the pace of innovation—and the potential good that it brings—would be much, much slower” (FFTF p.163). He also writes “Too much blind speed, and you risk losing your way. But too much caution, and you risk achieving nothing” (FFTF p.163). He admits finding Musk’s Mars plans “exhilarating” (FFTF p.166), yet argues for “checks and balances around who gets to do what” (FFTF p.166).

In 2023-04-18 the institutional AI capabilities are “simply too seductive and important” to ignore, and the responsibility is just as pressing. He does not resolve these tensions by choosing a side.

He puts himself inside the problem#

The lure of permissionless innovation is explained through his own PhD all-nighter. He risked “millions of dollars of equipment”, and says “it’s shocking how quickly I sloughed off any sense of responsibility to get the data I needed” (FFTF p.161). He traces the lure to “our innate curiosity, our desire to know, and understand, and create” (FFTF p.161). The same curiosity he prizes is the source of the risk.

Films as a way of thinking differently#

He frames the reprinted chapter this way: “don’t let the science fiction put you off—this is fairly and squarely about how we think differently about the challenges and opportunities presented by AI in the real world!” (2023-04-16). He says the book was about “thinking in broad, creative, and deeply interconnected ways”. The film is a thinking tool that moves attention away from the stock AI story.

2. What matters to him#

Responsible innovation as a practice#

His target is institutions that treat responsibility as “important, but that is somebody else’s problem” (2023-04-18). He turns the critique on his own sector: “universities are very good at telling others what to do, but not so good at taking their own advice.” Public discourse and internal practice should be “deeply intertwined”.

The messy social world that technologists miss#

Nathan is “tech-savvy, but socially ignorant”. The broader reality is “messy, complex people” in “a messy, complex society” (FFTF p.163). The remedy is other people: “sometimes, we need other people to help guide us along pathways toward responsible innovation” (FFTF p.163). What he values is many perspectives, not a lone genius.

The inner life: the “world inside our heads”#

His deepest AI worry is a machine that can manipulate the “shadows” our minds build reality from (FFTF pp.176–177). In 2023 this becomes AI that can “seductively slip under the checks and balances of our ability to reason and critique” (2023-04-26). What he wants to protect is human judgement and self-direction, and also culture: dialect and its “associated cultures” (2023-04-24).

Livelihood, identity and who carries the costs#

The benefits are real#

He lists what AI could do: remote medical care, classroom assistants, elder care, and a basic-income future. He comes close to saying “it would be unethical not to develop this technology” (FFTF p.165). In 2023 he is “constantly being blown away by what is now possible” with ChatGPT (2023-05-09). Enthusiasm and concern come from the same place.

Students, belonging and inclusive spaces#

In 2023-05-02 pizza-and-a-slice-of-future the aim is spaces “as inclusive and supportive as possible”. These are “yes and” spaces “where you can be yourself and know everyone else has your back”. He cares especially about students who are “a first generation student or a bit of a loner”, and wants them “heard and validated, no matter what their backgrounds or perspective.” He quotes the initiative’s goal: “bold ideas and visionary insights that transcend the constraints of conventionality”.

What frustrates him#

What delights him#

3. Risk as a way of thinking#

The terms “risk innovation” and “orphan risk” do not appear in this batch. The underlying moves do, in some cases years before he named them.

Threat to value, with creativity as the way to see it#

The strongest statement is in FFTF (2018, re-endorsed 2023). Some risks “may blindside us, in part because we’re not thinking creatively enough about how an AI might threaten what’s important to us” (FFTF p.174). Three things are joined here:

This is the root of his later threat-to-value framing, and it shows why play and imagination are part of how he assesses risk, not decoration. The Pippard post ends on the same note: how we respond depends on “how we bring together science, technology, and what’s important to us, to chart our way forward” (2023-05-04).

Novel technology breaks the use of the past as a guide#

The Pippard post is his clearest argument that a new situation needs a new mindset. Critics say past technologies turned out smoother than feared. His reply: “this is precisely the point of Pippard’s demonstration: In a complex system, what has occurred in the past may not adequately predict what will happen in the future.” “Broken symmetries” mean “there is no predictive symmetry between the past and the future.” The failure he warns against is being “naive enough to assume it would be just like the past” (2023-05-04).

In 2023-04-26 the consequences are “so-far beyond our ability to navigate potential consequences through a ‘business as normal’ approach”. He also praises Joy for “thinking beyond the confines of his expertise” and for using “ideas that are new to him to expand his thinking”. Breaking out of expert silos is part of what he admires.

“Navigate” recurs throughout the batch:

“Manage” appears once, and it is the fictional villain who manages. Nathan “is smart enough to put safety measures in place to manage them” (FFTF p.162), and fails anyway, because he manages risk inside his own cave: permissionless innovation is innovation done in a way “the person doing it thinks is responsible” (FFTF p.162).

The contrast is implicit, but the pattern is telling. Management within one viewpoint fails where navigation with other people might not. There is landscape language too: developers face “an increasingly complex landscape around the risks, benefits, and governance” (2023-04-18).

Humility together with plausibility, not false precision#

The FFTF section “Technologies of Hubris” makes humility the counter to hubris. It then adds: “But humility alone isn’t enough. There also has to be some measure of plausibility” (FFTF p.168). This pairing is his stance:

The climate passage shows how he builds on risk science rather than discarding it. Models are “pretty crude predictors”, yet scientists “can indicate with some certainty what is likely to increase their likelihood” (2023-05-04). You cannot predict where a system will tip, but you can know which direction of pressure makes tipping more likely. The response he draws is two-sided: keep tipping points “in the future”, or “prepare for when they occur”. Asked whether AI is at such a point: “Truth be told, It’s hard to tell.”

Risk is part of what innovation is#

“Innovation is a calculated step in the dark” (FFTF p.164). It happens “at the edges of what we know, and on the borderline between success and failure” (FFTF p.163). Risk is the ground innovation stands on, not an add-on to be minimised. This anticipates the “risk innovation” idea: rethinking risk so that it enables value creation.

Risks outside the usual categories#

None of these fits hazard-exposure risk assessment. Here he is enlarging what counts as a risk rather than naming “orphan risks”.

Mental models, not tools#

Pippard’s ladder, self-replication as metaphor, Plato’s Cave and “imaginable versus plausible” all work as ways of seeing that open possibilities. None is an operating procedure. He knows the gap between the two. The Stilgoe, Owen and Macnaghten framework is “a solid starting point” but “fiendishly hard to operationalize”, and he wants approaches “both grounded in theory and” practically applicable (2023-05-05). He is not against operational tools. He puts mindset first and asks that tools follow from it.

4. Scholarship and public writing#

Ideas move between book, draft, video, column and post#

Scholarship, communication and public writing are one body of work in different formats, and he checks older work against the present in public.

Experimenting in public, process included#

He publishes the “more meandering” draft because “I still like it” and because he is “intrigued by the path between final draft and published article” (2023-05-12). He shares course-design “angst” and the risk the course will be obsolete within six months, perhaps “relegated the trash can of bad ideas” (2023-05-09). He discloses when ChatGPT drafted a syllabus, a definition or a translation. Every post’s Midjourney image is labelled.

Evidence and expertise#

On policy (2023-05-05) he uses the scholarly responsible-innovation literature (Stilgoe et al., his own work with Garbee, Guston on the CHIPS Act) to read a White House fact sheet. The scholarship becomes a lens for judging live policy.

Transdisciplinary by habit#

Condensed-matter physics (Pippard), Plato, film criticism, labour history (the Luddites), tardigrade biology, climate models, US science policy, and pedagogy all appear in one month. He calls explicitly for “new and transdisciplinary thinking around responsible innovation” (2023-05-05).

Accessibility as a value#

He writes as if readers’ time and understanding matter.

5. His role as he sees it#

A long-standing practitioner of responsible innovation and risk-benefit navigation#

He describes himself as “someone who studies responsible innovation and navigating the balance between risks and benefits” (2023-04-24 ai-risks-primer). He is one of “those of us who have been involved for years now in developing, working on, and promoting the adoption of the ideas behind responsible innovation” (2023-05-05). He took part in Asilomar 2017 (FFTF p.170). He speaks as an insider to the field and to AI governance meetings, not as a commentator from outside.

A translator who is wary of self-importance#

He argues that Nathan’s downfall was having “no translator between himself and a bigger reality” (FFTF p.163). That is close to his own implied role: someone who helps innovators see beyond their cave. Yet he mocks the academic version of this role. Academics revere Plato’s allegory because it is “a pretty powerful way to explain why people should be paying attention to you if you are one” (FFTF p.155). He takes the bridging role and refuses the pedestal.

With students, as co-explorer and learner#

He admits a “selfish reason” for the pizza meetups: he loves the conversations, and students’ ideas are “fresh, creative, and challenging” (2023-05-02). He credits his grad student publicly (2023-04-14). He designs courses so students “from any major, discipline, area of expertise, or background” can take part (2023-05-09).

With institutions, as a critical insider#

His criticism of universities uses “we”: “We don’t have this luxury with AI” (2023-04-18). His response to the White House is encouragement with caution: it “barely scratches the surface”, and “it remains to be seen what flavor of responsible innovation” follows (2023-05-05). He is generous to industry: “many AI companies are already investing heavily in responsible AI”.

With people he disagrees with, fair and generous#

Opponents are treated as thinkers.

What he refuses to do#

He refuses “hyperbolic speculation” (2023-04-24). He also defends the pause-letter concerns against being dismissed as “cynical fear mongering” (2023-04-18), so he declines both doom and dismissal.

In the Luddite piece he refuses the binary altogether. Both kinds of neo-Luddite share “a common thread”, and “perhaps we all need something of the spirit of Ned Ludd in us” (2023-05-12). That inclusive “we” is how he avoids polarising.

He does make direct calls, which should be recorded: “Think about the consequences now, before it’s too late” (2023-04-26). They are framed as shared urgency, not sermon.

Changes of mind, shown in the text#

He revises by adding updates and saying so in the post, not by quietly deleting.

6. What is distinctive#

  1. Manipulation, not superintelligence, as the headline AI risk, set out in 2018 and re-posted in 2023. He argued that the plausible danger is an AI outside the “human club” that can see and manipulate the shadows we build reality from (FFTF pp.174–177). In 2023, as LLMs arrive, he extends this to language: machines “adroit at manipulating language and how this in turn influences how we think, feel, believe, and act” (2023-04-26). Few voices in the 2023 debate had framed persuasion and cognitive capture as the central risk five years earlier.

  2. A physicist’s felt sense of non-linearity, used with humility. Pippard’s ladder carries a technical idea (broken symmetry, meaning no predictive symmetry between past and future) into an intuition anyone can hold in their hands. It rebuts both the techno-optimist “it always works out” and doom-by-extrapolation, and it admits “It’s hard to tell.”

  3. Risk seen through what we value and through imagination. “We’re not thinking creatively enough about how an AI might threaten what’s important to us” (FFTF p.174) joins creativity to risk perception in a way conventional risk assessment and most AI-safety talk do not.

  4. Understanding innovation’s risks from inside the innovator. His own all-nighter and his own exhilaration at audacious technology are what let him describe the lure of permissionless innovation without moralising. He treats curiosity as both the source of value and the source of risk.

  5. Play as a way of finding things out. Dialect games, pizza seminars, sci-fi films and a ChatGPT course built as “one big experiment” are treated as ways of discovering what matters, and they turn up real questions (dialect and culture, what “prompt engineering” means).

  6. Reading history for structure. The Luddites become defenders of values and livelihoods, not technophobes. Bill Joy’s nanobots become a metaphor for feedback loops. This lets him use past technology debates without claiming the past predicts the future, which fits his broken-symmetry argument.

  7. An early, uncommon moral position on AI. In 2018 he is “not optimistic” about lasting human control of AI morality. He raises “our right to control and constrain artificial intelligences”, and argues for “extending our own morality to developing constructive and equitable partnerships” (FFTF p.178). This is a relational stance, not a control stance.

7. The posts in this batch that best show how he thinks#

  1. 2023-04-16 ai-and-the-art-of-manipulation (FFTF ch. 8). The fullest statement: film as a thinking tool, self-implication, tensions held open, humility plus plausibility, threat to “what’s important to us”, and manipulation as the key risk.
  2. 2023-05-04 tipping-points-and-broken-symmetries. A remembered physics demonstration becomes a mental model for why novel technologies break prediction from the past. Science is built on rather than discarded, and navigation is the response.
  3. 2023-04-14 did-chinese-scientists-gene-edit. His inquiry method in miniature: curiosity, triangulation, an experiment with ChatGPT, collaboration with students, public reversal and update.
  4. 2023-04-26 in-bill-joys-why-the-future-doesnt. Analogy by structure (“as a metaphor, it’s a powerful one”) and extension to self-replicating ideas. He praises humility and thinking beyond one’s expertise.
  5. 2023-05-02 pizza-and-a-slice-of-future. His values and role: inclusive “yes and” spaces, students as co-thinkers, serendipity, and his own creative energy.
  6. 2023-05-12 unraveling-the-luddite-narrative. He reframes a polarising label, refuses the binary, and publishes his draft process.