Late Lessons, Jensen Huang and AI
Markdown version

FFTF-A perspective notes: how Andrew Maynard thinks, from Films from the Future (2018)#

What was read. - Read in full: book/FFTF-A.txt. This covers ch1 “In the Beginning” (pp.14–26), ch2 “Jurassic Park: The Rise of Resurrection Biology” (pp.27–45), ch14 “Looking to the Future” (pp.287–291) and the acknowledgments (p.292). - Skimmed for method: FFTF-B to F, which cover chapters 3–13. For each chapter I read the opening and closing, plus every section where he describes his own stance, method or history. - Read almost in full: ch8 Ex Machina, because it is the book’s main AI chapter. - Read closely: selected sections of ch9 Transcendence, ch10 The Man in the White Suit, ch11 Inferno, ch12 The Day After Tomorrow and ch13 Contact.

Evidence rules. - The book is sole-authored (Mango, 2018), so all of the text counts as evidence of his thinking. - Page numbers are the printed FFTF pages, which match the PAGE markers in the text files. - I did not open the earlier notes-FFTF-*.md files in book/, or any of the excluded project files.

Dating. - He wrote the book in 2017–18. Hurricane Harvey is happening “as I’m writing this” (p.42), and he is writing the last chapter on Arran “nearly thirty-four years” after 1984 (p.287). - The book comes before the Future of Being Human initiative and before generative AI. His views on AI here are 2018 views, and I date them as such.

The main finding for this batch. Films from the Future is neither a movie book nor a risk-management manual. It shows a way of thinking, and he says so: - He set out “to focus on how we think about technological innovation, society, and the future” (p.291). - “there are no easy guidelines or rules of thumb”, and “much of this book is devoted to ways of thinking that reduce the chances of making a mess of things” (p.39).

At the centre is a pairing that earlier readings could easily miss. His reframing of risk as a threat to what people value is tied, as method, to his use of story: - “risk is at the core of all the movies here” (p.23). This is because dramatic tension is built from what characters stand to lose, or cannot bear to stop hoping for. - So films are where he trains himself and his readers to see risks that conventional risk thinking cannot see: to “dignity, belonging, identity, belief, even what it means to be human” (p.23). - Imagination lifts us “out of the rut of conventional thinking” (p.24). Critical thinking and a test of plausibility keep that imagination honest (p.282).

Almost every element of his September 2026 self-account is already in print here in 2018: - Conventional risk thinking runs out of steam. Established approaches “run out of steam rather fast when we’re facing technologies that can achieve things we never imagined” (pp.22–23). - Risk Innovation is “where much of my current work lies” (p.23). - Navigating a risk landscape. He writes of the need to “navigate a radically shifting risk landscape” (p.23). - Risk as a threat to value (pp.23–24). - Humility against hubris. This runs through the whole book. - Delight, serendipity and curiosity. He names them (p.18).

Play is mostly something he does rather than something he names. It shows in how he treats films, readers, students, and his own ideas.

1. How he thinks#

He starts inside the story, usually with himself in it, and often at his own expense. Each chapter he wrote opens with a scene rather than a thesis: - Age sixteen, on 1 January 1982, watching 2001 on a black-and-white portable TV while his parents entertain guests. His message to his younger self: “Take note—this is important”, and also “Don’t be such a jerk” (p.14). - As a new PhD, going to Jurassic Park because he and his wife “fancied a night out” (p.27). - On Arran, in a rental car “that modern automotive technologies seem to have completely bypassed” (p.287). - As a child, missing Sagan’s Cosmos because his parents kept getting rid of the TV: “Sadly, I wasn’t one of them” (p.271).

This is not decoration. It puts him inside the problem as someone who is also learning, not above it as an authority.

He treats a film as a “jumping-off point”, not as a prediction or something to fact-check. - The book is “not a book about great science fiction movies, but a book about how science fiction movies can inspire us to see the world around us and in front of us differently” (p.16). - He values films “not because they are accurate or prescient, but precisely because they are not tethered to scientific accuracy” (p.288). Like all good storytelling, they “sometimes play around with reality to reveal deeper truths” (p.15). - He chose some films that “bombed with the critics” because they filled a gap in the larger story (p.17). - He chose films that were never meant as science fiction. For Ishiguro, the clones in Never Let Me Go were “simply a plot device” (p.47). The Man in the White Suit is a 1951 comedy about stain-resistant fabric (p.208). - His film summaries are, in his own words, “idiosyncratic, to say the least”. They show “what grabs my attention” (p.26). He picks each film for what it opens up.

The chapters follow the same pattern, and the pattern is itself his method: 1. A personal hook or a strange real-world case: Clonaid’s “Baby Eve” (p.46), the Veris “Trust Index” (p.63), Pleistocene Park (p.31), Neuralink (p.128). 2. The film, retold for what matters. 3. The real science, explained with homely analogies. The lysine safeguard is “about as useful as trying to starve someone by locking them in a grocery store” (p.38). Patching a genome with frog DNA is like losing pages from a skyscraper’s plans and slipping in pages from “a cookie-cutter duplex” (p.38). Our digital circuits are “about as useful to superintelligence as the brain cells of a flatworm” are to a theory of everything (p.171). 4. A deliberate complication of the obvious moral: “this is too simplistic a takeaway” (p.44); “There is a glitch in this argument, however” (p.163). 5. An open question. 6. A bridge to the next film. Every chapter ends by handing on to the next, so the book reads as a single “journey” (pp.16, 26, 291).

He reads films against the grain to find the less obvious risk. - In Jurassic Park, what is at risk is “John Hammond’s dream”. In Never Let Me Go it is “Tommy’s hope for the future”. In Ghost in the Shell it is Major Kusanagi’s “sense of who and what she is” (p.24). - For Inferno he turns to the novel’s different ending, a sterility virus instead of a killer one, to reach the harder question of consent (pp.247–249). - For Ex Machina he reaches back to Plato’s Cave, which is not in the film’s dialogue at all, to find the chapter’s real subject: what it means for something to control “the shadows on the walls of their mind-caves” (p.177).

He thinks across technologies, not in silos. - “It’s when they begin to converge that the really interesting stuff begins to happen” (p.18). - He organises the book around convergence across bio, cyber and materials technologies (p.17), not one technology at a time.

His physics training sets the terms: nothing is fully predictable, but outcomes have limits. He was an undergraduate physicist in the years of fractals and “strange attractors” (p.39). From chaos theory he takes two lessons together: - “we cannot wield perfect control over complex technologies within a complex world” (p.41). - There are nonetheless “boundaries” that separate “plausible futures from sheer fantasy”, and “points of stability”. Some futures are “more likely to occur if we take the appropriate courses of action”, and some “can be squandered” (p.41).

That combination (no control, but some bounds and some leverage) is the logic behind navigation as opposed to management or prediction. He applies it again to predicting behaviour: we will “draw boundaries around more or less likely behaviors”, but never predict with certainty (p.76).

He makes the strongest case for the side he does not take, and puts himself in the argument. - De-extinction. The critics “have a point”, but “some would argue that not to use them would be verging on the irresponsible” (p.35). - Power. It should not be abdicated, “The challenge we face is not to abdicate power, but to develop ways of understanding and using it in ways that are socially responsible” (p.44), and profit-driven innovation “has also created a lot of good” (p.44). - Permissionless innovation. He admits it excites him: “I must admit that I find this exhilarating”, of Musk’s Mars plans (p.166). “In the world of Nathan, he’s a hero” (p.164). - The Luddites. He sides with them as people protecting fair livelihoods: “If Elon Musk is a Luddite, count me in!” (p.191). - The AI executive who refused to engage the public “was right to be concerned”, even though Maynard thinks not talking is the greater risk (p.227). - His resolution is “yes and”, not either/or (p.269).

He tests ideas on himself, and turns lessons into games. - Sent a “Minority Report-like” trustworthiness test, “Naturally, I took the test. I got a Trust Index of nineteen”. He then got colleagues to take it and found “academics are some of the most felonious people around” (p.64). Poking at a claim yourself, with humour, becomes a critique of biased training data. - In his Entrepreneurial Ethics course he ran a no-rules art-trading game for a $25 Starbucks voucher. It showed how fast stated values collapse under incentives. His footnote: “nothing is ever ‘just a game’” (p.221). - While writing about smart drugs he searched for “humility pills”, “thinking how telling it is” that people want to be smarter but not humbler. To his surprise he found a real paper with that title (p.108 n.65).

Humour carries the argument; it is not a break from it. Examples: - Smartphones: “They’ll even allow you to make phone calls” (p.19). - Superintelligence evangelists at Asilomar: “I sometimes had to remind myself that I was at a scientific meeting, not a religious convention” (p.170). - His own aerosol paper’s citation count: “at least two of those ‘fans’ were me” (p.219 n.151). - The book “fails miserably on the ‘where to get the best drinks’ front” (p.290).

The lightness is how he refuses to preach. It is also how he models the “Don’t Panic” temper he recommends (p.289).

2. What matters to him#

3. Risk as a way of thinking#

He places himself in risk science and names its limits in the same passage (pp.22–23). - His background: “Most of my professional life has been involved with risk”. It includes particle-inhalation research, nanotechnology risk, teaching risk assessment, and running risk centres. - His frustration: “I have less and less patience for how many people tend to think about risk” (p.22). - His diagnosis is specific: established approaches “work reasonably well” for conventional technologies but “run out of steam rather fast when we’re facing technologies that can achieve things we never imagined” (pp.22–23). - His image: “squeeze the new wine of technological innovation into the old wineskins of conventional risk thinking” (p.23). - His prescription: “realign how we think about risk with the capabilities of the innovations we’re creating”, and radical innovation “in how to think about and act on risk” in order to “navigate a radically shifting risk landscape” (p.23). He points readers to his 2015 Nature Nanotechnology piece, “Why we need risk innovation” (p.23 n.3).

Note what this is not. It is not a rejection of quantitative risk assessment, which is his own training and which “work[s] reasonably well” where it fits. It is a claim that the frame has to grow when the technology is new in kind. That is exactly the “change our whole mindset” move in his self-account.

Risk as a threat to value, stated in full, and presented as a way to open up new thinking: - Films “explore other, subtler risks, including threats to dignity, belonging, identity, belief, even what it means to be human” (p.23). - These risks “get at what is so important to us that our lives are diminished if it’s denied us, or taken from us” (p.23). - The definition covers aspiration as well as possession: “whether it’s something we have and can’t face losing, or something we aspire to and cannot bear to lose sight of” (pp.23–24). Few risk frameworks contain this. - It is framed as generative: “it’s this insight that opens up interesting and new ways of thinking about the social consequences of technological innovation” (p.24).

Why story is the method. “Each of these films has a risk-based narrative tension that keeps its audience hooked” (p.23). Stories are built from threatened value. That makes them a precise tool for surfacing value-based risks that a hazard-and-exposure frame misses: - Tommy’s hope (p.24). - Kusanagi’s identity (p.24). - Hammond’s dream (p.24).

The same reasoning explains why he says, “I’m a sucker for using the imagination in science fiction movies to stimulate new ways of thinking about risk” (p.24).

He reads resistance as protecting value, not as ignorance. This is a steady interpretive habit, and it matters a great deal for public reactions to AI: - In The Man in the White Suit, the resistance “is not a resistance to technological innovation, but a fight against something that threatens what is deeply important to the people who are resisting it”. “Everyone is shrewd enough to see how change supports or threatens what they value” (p.225). - The Luddites fought “not against technology, but against its socially discriminatory and unjust use” (p.192). - Hostility to “not-normal” augmented bodies arises when “different” is seen as “threatening something they value” (p.139). - Resilience itself is redefined as “not necessarily about maintaining the status quo, but about protecting and preserving what is considered to be ‘of value’” (p.262).

Complexity: you cannot control everything. - Perrow’s “normal accidents” (p.43). - The Arkema plant during Harvey, where the cascade included “a snake, washed in by rising waters”. “Overflowing toilets and snakes? Probably not” (pp.42–43). - “If Hammond had read his Perrow” (p.43). - The warning is against “thinking you’re smart enough to have every eventuality covered” (p.38), against the hubris of control. Scientists are “tripped up by what they don’t know, and what they don’t care to find out” (p.38).

Plausibility is how he disciplines imagination. He polices hype and fear alike: - “Imaginable” versus “plausible” (p.170). - Exponential extrapolation is “dangerously sensitive to the assumptions that underlie” it (p.199). - “when make-believe is treated as plausible reality… actions … end up harming people” (p.205). Either through fear (techno-terrorism, blocking useful technologies) or through lost benefits: “what a tragedy it would be if we turned away from some technological futures that could transform lives for the better, simply because we become confused between reality and make-believe” (p.206). - Occam’s Razor helps set priorities, but “should never be considered as more than an aid to decision-making” (p.281).

Humility about AI in particular, in 2018. - “we’re all looking into a crystal ball as we gaze into the future of AI, and trying to make sense of shadows and portents that, to be honest, none of us really understand” (p.170). - On superintelligence: “Here, I freely admit that I may be wrong” (p.170).

How the 2018 book applies risk-as-value to AI (ch8): - The key sentence: plausible AI risks “may blindside us, in part because we’re not thinking creatively enough about how an AI might threaten what’s important to us” (p.174). Creativity is treated as a skill of risk thinking. - The risk he ranks highest is manipulation, not domination: “the ability of future machines to bend us to their own will” (p.174). We “need to worry less” about superintelligence “and more about guarding against AIs that learn how to use our cognitive vulnerabilities against us”, and we need “tests that indicate when we are being played by machines” (p.177). - He sets aside the definitional question: “It’s not clear whether this behavior constitutes intelligence or not, and I’m not sure that it matters” (p.176). - Related risks elsewhere in the book: - Algorithmic bias and opacity: “artificial brains that we are increasingly ignorant of the inner workings of” (p.78). - The values built into our definitions of intelligence: “if we start off with a warped perspective of intelligence and success… the ‘intelligence-enhancing’ technologies we develop… will be equally warped” (p.108). - His long-run view is partnership rather than control. “I’m not optimistic about this level of human control over AI morality in the long run”. The way forward is “extending our own morality to developing constructive and equitable partnerships with something that sees and experiences the world very differently from us”, perhaps “artificial emissaries” acting as “machine-philosophers” (p.178).

Dated position to check against later writing. In 2018 he called Bostrom-style superintelligence “currently scientifically implausible” (p.171). He also said Occam’s Razor would “probably favor” spending on harm from new materials over “preparing for the advent of superintelligence (both of which depend on a house-of-cards stack of assumptions)” (p.281, where the other example is gray goo). He hedged openly (“I may be wrong”, p.170). His later work should be checked to see how, or whether, this changed. The method (imaginable versus plausible, openly held uncertainty) is more lasting than the verdict.

4. Scholarship and public writing#

His public writing is part of his scholarly record, and he cites it that way. Footnotes to his own work sit alongside Nature, Research Policy and Perrow: - 2020 Science blog posts (p.85 n.45, the 2009 list of technology trends that put nootropics at number nine; p.191 n.128). - The Conversation op-eds (p.123 n.76; p.159 n.104; p.165 n.107). - A Slate piece (p.171 n.115). - His Nature Nanotechnology commentary (p.23 n.3).

There is no line between “the research” and “the outreach”. The blog is where the idea was first tested, and the book is where it is gathered.

He uses his platform to bring critics in. In 2009 he invited colleagues from civil-society organisations, including Jim Thomas of ETC, to write for 2020 Science. He knew their “sometimes critical stances” and “wanted to get a better understanding of how they saw the emerging relationship” (p.191). Public writing, for him, is a convening act, not only a broadcasting one.

He brings stories back from closed rooms into public view. Many of the book’s teaching moments come from elite settings, retold without names: - A World Economic Forum session where “to my surprise” someone proposed art (p.25). - Dinner at Asilomar being asked “So, do you believe in superintelligence?” (p.170). - A prominent scientist at PCAST answering that public engagement “sounds like a very bad idea” (p.222). - An irate scientist at a National Academies workshop (p.223). - An AI executive who would not talk to the public (p.226).

This is how an insider’s access becomes public understanding.

Accessibility is a principle, not a watering-down. - Films can be appreciated “as much by someone who flunked high school as by a Nobel Prize winner” (p.18). The book is written to the same standard. - “No one ever reads an overlong, overweight, and utterly incomprehensible guide” (p.290). - He starts Risk Bites on YouTube in 2012 (p.126 n.80) and calls academia’s absence from YouTube “a glaring missed opportunity” (p.127). He is honest that he is “somewhat leery of YouTube, despite using the platform extensively myself” (p.126). - He links this to his institution. ASU is trying to make knowledge “as accessible, impactful, and socially relevant as possible”. “It’s why I work here” (p.125 n.79).

Rigour stays visible. - He corrects his films in the footnotes. DNA from amber is “Sadly… not a realistic” premise (p.29 n.5). Chaos theory in Jurassic Park comes with “a lot of hokum” (p.41). - He recommends scholarship to lay readers, for example Stilgoe, Owen and Macnaghten on responsible innovation (p.22 n.2). - He admits the limits of the whole book: “a very incomplete guide” with “absolutely no mention of blockchain” (p.291).

He drafts in company. He named more than twenty colleagues who read drafts and gave “critical feedback that reduced my chances of making a fool of myself” (p.292).

Engagement is plural and informal: “science museums, TED talks, science cafes, poetry slams, citizen science” (p.228). People are met “where they’re at” (p.229).

5. His role as he sees it#

He names the role himself: Honest Broker. On Pielke’s typology: “it’s clear from his writing that he’s a fan of the honest broker. And, to be honest, so am I. This is the role I try to carve out for myself in my public-facing work, trying not to judge others or advocate for a specific course of action, but to help people make the best-informed decisions for themselves and their communities” (p.246). - Why: it “avoids mistaking personal values for the ‘right’ values, and respects deeply held beliefs and values in others, even where you may disagree with them” (p.246). - He also names its limit. “I’ll be the first to admit that this role… has its problems”: on issues like climate or vaccines, not advocating can become “tacit support for not taking action” (p.246). There he prefers collective, institutional advocacy and the weight of the evidence to lone voices (pp.244–247). - This explains his refusal to preach or fear-monger. It follows from respect for other people’s agency and from being aware of his own fallibility.

A guide on a journey, not an oracle. - The closing model is Adams’s Hitchhiker’s Guide: “neither claims to be a comprehensive, infallible, all-encompassing guide” (p.290). - Its motto, “Don’t Panic”, is “as good a piece of advice as any”, held together with “Of course, we shouldn’t be complacent—far from it” (pp.289–290). - He says he is “optimistic enough to believe that we have the collective ability to develop new technologies in ways that work for us, not against us” (p.290).

Opening conversations across divides, without dismissing expertise. - Art offers “a common point of focus” in a world “so deeply and divisively divided along ideological lines”. It works against “our tendency to close down our imagination (together with our humility and empathy)” (p.25). - He is careful not to go populist: expecting a random person to engineer organisms safely “would be crazy”. “But one thing we’re all qualified to do is think about what the possible consequences of technology innovation might mean to us and the people we care for” (p.26). - “Most people have a pretty high level of expertise in what’s important to them and their communities” (p.222).

A scientist speaking to scientists, with sympathy. - “At heart, I must confess, I’m still a technology geek” (p.17). - “I’ve met remarkably few scientists and engineers who would consider themselves to be unethical or irresponsible” (p.36). - He admits the same flaws he describes in others: - “Benevolent myopia I see in many of my peers, and even myself at times” (p.220). - His PhD all-nighter risking “millions of dollars of equipment”: “Looking back, it’s shocking how quickly I sloughed off any sense of responsibility to get the data I needed” (p.161). - His aerosol paper, where he was “quite happy to coopt a narrative of social good” to satisfy his curiosity (p.219). - “All of us, it has to be said, have a bit of Sidney Stratton in us” (p.227).

His critique comes from inside the community and includes himself, which is why scientists and entrepreneurs can hear it.

Self-aware about the “enlightened translator” pose. He jokes that academics love Plato’s Cave because “it’s a pretty powerful way to explain why people should be paying attention to you if you are one” (p.155). Yet he argues Nathan’s downfall was having “no translator between himself and a bigger reality” (p.163). He knows the helpful go-between role exists, and he knows how easily it turns into self-importance.

Why he wrote the book: to help people “wrap our collective heads around what’s coming our way” (p.288), and because “we collectively need to give a damn about the future we’re creating… it’s partly why I wrote this book” (p.286).

6. What is distinctive#

  1. He ties a value-based idea of risk to a narrative method. Many people call for broader risk framing. Maynard uses story, whose engine is threatened value, as the working tool for seeing it (pp.23–24). Few risk scientists work through film. Few film or tech commentators bring a background in quantitative exposure science. He has both, and he treats creativity as a risk skill: we may be “blindsided… because we’re not thinking creatively enough about how an AI might threaten what’s important to us” (p.174).

  2. He holds two disciplines together. Imagination, freed from accuracy, opens things up. A physicist’s sense of plausibility closes off fantasy. “Critical thinking alone is almost inhuman in its cold impartiality. On the other hand, creativity on its own leads down a path of fantasy and delusion. But when the two are combined…” (p.282). This is his clearest statement of how he knows things. It lets him stand against both hype and doom, and against “meh” (pp.284–286).

  3. He navigates rather than manages or predicts, and the reason comes from physics. Chaos gives him unpredictability within bounds, points of stability, and futures that can be “squandered” (p.41). “Navigate” recurs throughout (pp.15, 18, 21, 23, 86, 107). “Assessing and managing” is the language he gives to the older approach (p.23).

  4. He reads resistance charitably. Public backlash is information about what people value, not scientific illiteracy (pp.192, 225). That changes how “public concern about AI” should be read.

  5. He holds a two-sided ethic: equity plus a duty to innovate. He will not renounce technology “from a position of privilege” (p.288), and he will not accept innovation that lets people “slip between the cracks” (p.18).

  6. His AI instincts were early. In 2018 he put manipulation of human cognitive weak spots ahead of superintelligence (pp.174–177). He called for tests of “when we are being played by machines” (p.177). He tied AI risk to the values built into how we define intelligence (p.108). He imagined partnership with an intelligence that “sees and experiences the world very differently from us” (p.178). Well before persuasive language models, his lens of value, story and plausibility located the harm in the relationship between AI and human minds.

  7. He is humble and implicates himself, and he does not preach. He confesses his own myopia and his excitement about innovation. He admits he “may be wrong”. He treats entrepreneurs, activists and sceptics fairly. He chooses the honest-broker role on purpose (p.246). His humour does real work. This is a stance many AI commentators do not have, and it lets him reach people across camps.

  8. Awe is a civic resource. He treats the loss of wonder as a hazard: complacency hands the future “to those that do care the opportunity to do what they like” (p.286). Delight is part of the argument, not a garnish.

Absences and limits in this text. - “Play” is not named as a principle. It is practised everywhere: games, jokes, “being a little playful with Bostrom’s ideas” (p.169), films that “play around with reality” (p.15). - “The future of being human” is foreshadowed by “what it means to be human” (p.23) and the Ghost in the Shell chapter title, but it is not named. - The book gives no operational method. He says there are “no easy guidelines or rules of thumb” (p.39). That fits his self-account: ideas that open up possibilities, not procedures.

7. The most revealing passages#