Notes: Films from the Future (2018), batch FFTF-A#

Source: working/maynard/book/FFTF-A.txt, read in full. It holds Ch1 “In the Beginning” (pp. 14–26), Ch2 “Jurassic Park: The Rise of Resurrection Biology” (pp. 27–45), Ch14 “Looking to the Future” (pp. 287–291) and the Acknowledgments (p. 292).

Page convention: page numbers are the PDF page markers, which match the printed folios (e.g. the page marked 23 carries the printed “23”). Where a quote runs across a page break, both pages are given.

Provenance: all running prose is Andrew’s. The following are not his words and are not used as evidence of his views: the chapter epigraphs (HAL, p. 14; Ian Malcolm, p. 27; Hitchhiker’s Guide, p. 287), Malcolm’s “could/should” line (p. 36), and the Buzzfeed logbook quotation (pp. 42–43). His choice of these lines and how he frames them are noted where they matter. The plot summaries are his own, but they are described in the book as “idiosyncratic” (p. 26) and are used here only where they carry analysis.

Chapter 1: In the Beginning (pp. 14–26)#

Argument#

This chapter sets out what the whole book is for and the lenses it uses. Science fiction films, “viewed in the right way—and with a good dose of critical thinking” (p. 15), are a way to see “around the corner of our collective near future” (p. 15). They help people think about the social consequences of technologies “we don’t yet have, but that are coming faster than we imagine” (p. 15). Their value lies in not being tied to technical accuracy: “these are stories about our relationship with the future” (p. 15).

Three threads are announced, and they run through the book:

1. Technological convergence: the really consequential developments happen when technologies merge (pp. 18–21).
2. Socially responsible innovation: the tension between developing technologies responsibly and “ensuring they reach their full potential” (p. 21).
3. Risk, rethought: conventional risk thinking cannot cope with emerging technologies (Risk Innovation, p. 23). Risk should be understood as threats to what people value, including “dignity, belonging, identity, belief, even what it means to be human” (p. 23).

A fourth thread is inclusive public deliberation. Films and art more broadly can “break down the barriers between ‘experts’ and ‘non-experts’” (p. 24) and act as “a common point of focus” (p. 25) in an ideologically divided world. Everyone has standing: “one thing we’re all qualified to do is think about what the possible consequences of technology innovation might mean to us and the people we care for” (p. 26).

The stance is enthusiastic but conditional. He calls himself “at heart, I must confess, I’m still a technology geek” (p. 17), yet says “this stupendous technological power comes with a growing obligation to learn how to handle it responsibly” (p. 17).

Concepts and frameworks (his definitions)#

• Science fiction as a “jumping-off point” (p. 16). The book is “not a book about great science fiction movies, but a book about how science fiction movies can inspire us to see the world around us and in front of us differently” (p. 16). Films reveal “hidden connections between who we are, the society we live in, and the technologies we create” (p. 16). They “remove barriers to people talking together about the future” and slip “past our preconceived ideas” (p. 18).
• Technological convergence. He defines it as “what happens when different strands of innovation intertwine together” (pp. 18–19). The iPhone is his exemplar, a set of devices “more than the sum of their parts” (p. 19). He adds that “Smartphones are a useful, but still rather crude, example” (p. 19). The deeper convergence is “our growing ability to blur the lines between physical technologies” (materials, machines, electronics), biological technologies (gene editing, biomanipulation) and cyber technologies (machine learning, natural language processing, massive-scale data) (p. 20). Its significance: “For perhaps the first time, we are getting close to being able to far outstrip nature in how we design and engineer the world around us” (p. 20). The power and the danger sit in the same place: “This is where the true transformative power of convergence lies, and it’s also where some of the greatest potential pitfalls are” (p. 20). He also notes a mechanism of acceleration: “technical knowhow from one area is used to solve problems and accelerate progress in another” (p. 19).
• Socially responsible innovation (p. 21), illustrated by Sidney Stratton in The Man in the White Suit. Stratton’s failure: “He never bothered to ask anyone else what they thought of his invention” and “made the classic mistake of thinking that, just because he could do something, others would love it” (p. 21). Elsewhere he pairs “socially responsive and responsible innovation” (p. 24) and “socially responsible and responsive development” (p. 26). Responsiveness is part of the concept.
• Responsible Innovation / Responsible Research and Innovation (RRI) (p. 22). His gloss: “we don’t always have a second or third chance to get things right when developing new technologies.” It is therefore better to think about consequences early and act early. Formal definitions boil down to “ensuring that anyone who is potentially impacted by technological innovation has a say in how it’s developed and used,” and to steering toward a better future “for as many people as possible, without causing undue harm” (p. 22). His verdict: “easy to say, of course, but fiendishly difficult to put into practice” (p. 22). He recommends Stilgoe, Owen and Macnaghten (2013) as the working definition (p. 22, n. 2).
• Risk Innovation (p. 23), “where much of my current work lies.” The argument runs as follows. Since the Industrial Revolution “we’ve become quite adept at developing new ways of causing harm,” and equally adept at assessing and managing the risks of mining, manufacturing, new chemicals and materials, and pollution. But “these approaches to risk belong to a different world than the one we’re now creating” (p. 23). Hence the core claim: “in order to navigate a radically shifting risk landscape, we need equally radical innovation in how to think about and act on risk” (p. 23). He cites his 2015 Nature Nanotechnology piece “Why we need risk innovation” (p. 23, n. 3).
• “New wine … old wineskins” (p. 23). “we’re in danger of desperately trying to squeeze the new wine of technological innovation into the old wineskins of conventional risk thinking, and at some point, something’s going to give.” Conventional approaches “run out of steam rather fast when we’re facing technologies that can achieve things we never imagined” (pp. 22–23). The remedy: “we need to realign how we think about risk with the capabilities of the innovations we’re creating” (p. 23).
• Risk as threat to value. His expanded definition: risk is “a way of thinking about risk that revolves around threats to what is important to us” (p. 23). That means “something we have and can’t face losing, or something we aspire to and cannot bear to lose sight of” (p. 24). It includes health, well-being and environment, but “also extends to less tangible but equally important things that we deeply value” (p. 24). These are “what is so important to us that our lives are diminished if it’s denied us, or taken from us” (p. 23). Two features stand out:
• It covers aspiration as well as loss: characters risk “either losing something of great importance to them, or being unable to gain something that they aspire to” (p. 24). Examples are Hammond’s dream, Tommy’s hope in Never Let Me Go, and Kusanagi’s identity in Ghost in the Shell (p. 24).
• It links risk to responsibility. Films “reveal subtle connections between irresponsible innovation and threats to what people value or aspire to” (p. 24).
• Art as “a common point of focus” (p. 25). This comes from a World Economic Forum meeting he moderated. The question there was how to ensure safe and beneficial technology “in a world that is so deeply and divisively divided along ideological lines” (p. 25). A participant proposed art. His development of the idea: “Art, in all its forms, is a medium that can mitigate our tendency to close down our imagination (together with our humility and empathy)” (p. 25). It comes with a guardrail: “as long as that imagination is grounded in reality where it matters” (p. 25).
• Everyone qualified, experts still needed (pp. 25–26). “Naturally, we still need technical experts, laws, and policies” (p. 25). A random person cannot safely engineer organisms or design aircraft, and expecting that “would be crazy” (p. 26). But everyone can judge what consequences mean for them and the people they care for (p. 26).

Themes#

• Risk. This is his professional home ground. He summarises his career: inhaled airborne particles, the health and environmental risks of nanotechnology, teaching risk assessment, and running risk centers (p. 22). He then says: “I have less and less patience for how many people tend to think about risk” (p. 22). He treats risk as the narrative engine of every film (p. 23) and widens it from death and ecosystems to dignity, identity and “what it means to be human” (p. 23).
• Responsible / socially responsible innovation. A named through-line of the book (pp. 21–22). The failure mode is not asking the people affected (Stratton, p. 21), not thinking about consequences, or “being too arrogant to see their blind spots” (p. 22).
• Permissionless innovation. The term is not used in this chapter. Its structural opposite is present: Stratton “never bothered to ask anyone else” (p. 21), and RRI means those affected get “a say” (p. 22).
• Hubris. The word is not used. The concept is carried by “arrogant” (p. 22) and “the folly of entrepreneurial arrogance” in Jurassic Park (p. 26).
• Developers’ mindsets. He resists blaming bad character. The difficulty “isn’t because scientists and engineers don’t care about who gets hurt—most of them care deeply—but because we’re charging headlong into a future that’s so complex, it’s becoming increasingly challenging to work out what could go wrong” (p. 17). Blind spots are structural. People “all too easily overlooked by scientists and engineers” must be involved, “in fact, especially these individuals” (p. 18).
• Governance and public engagement. Governance appears through the WEF discussion of “regulations and policies.” Its goal was to “nip problems in the bud without creating unnecessary roadblocks” (p. 25), which suggests a balanced view of regulation rather than a restrictive one. Investor pressure is another lever: JANA Partners and CalSTRS asked Apple to address iPhone impacts on teenagers (p. 19). Art and film complement technical expertise, law and policy; they do not replace them (p. 25). The democratic premise is that “we all have a role to play” (p. 18).
• AI and intelligence. AI is present but not yet central.
• HAL is his origin story. 2001 made him think about “a future where smart computers might decide that their self-preservation was more important than the humans who created them” (p. 15).
• A strong capability claim for 2018: “We are already creating artificial intelligence systems that can operate faster and smarter than any human” (p. 17).
• AI is listed as one strand of convergence (pp. 19–20). Transcendence shows AI with bio-, neuro- and nanotechnology, and mind-uploading (pp. 19–20).
• The trajectory is open: “we’re not sure yet whether the rapid development of artificial intelligence is going to make the world a better place or lead to the end of humanity as we know it!” (pp. 20–21).
• Automation’s effect on jobs is an open risk (p. 20).
• Convergence. A headline concept (pp. 18–21). The key epistemic point: “while convergence is massively accelerating our technological capabilities, we still have little if any idea what might go wrong, or what the unintended consequences could be” (p. 21). He says convergence is “a theme that runs deeply through this book,” and it “gets to the heart of the morality and the ethics” of technology (p. 21).
• Equity. The aim is to steer innovation “toward more beneficial and equitable outcomes” (p. 15). His concerns include harm to “communities that all too easily slip between the cracks” (p. 18) and access to technology meaning “the poor getting poorer and the rich richer” (p. 20).

Analogies across technologies (Ch1)#

Chapter 2: Jurassic Park — The Rise of Resurrection Biology (pp. 27–45)#

Argument#

He reads Jurassic Park as a film about “greed, ambition, genetic engineering, and human folly” (p. 28), and he identifies three lasting lessons beyond de-extinction:

1. The complexity of “using powerful new technologies in an increasingly crowded and demanding world” (p. 30), especially “where mixing people and technology together leads to unpredictable results” (p. 30).
2. The “could vs should” question: “when we should tinker with technology and when we should leave well enough alone” (p. 30).
3. “the sometimes oversized roles mega-entrepreneurs play in dictating how new tech is used, and possibly abused” (pp. 30–31).

The chapter then works through four movements.

(a) De-extinction and biology by design (pp. 31–36). He introduces Pleistocene Park and the Zimovs’ climate-stabilising rewilding, and George Church’s “woolly mammoth 2.0” (p. 34). He treats Church’s approach as closer to new species design than resurrection. That raises the question “why recreate the past, when we could reimagine the future?” (p. 34). He holds both sides. Critics who want to dial down human interference “have a point” (p. 35), yet “we cannot ignore the possibilities” (p. 35) of reading, recoding and “download[ing]” genomes back into the world (p. 35). He then escalates through new species, new DNA and alien life (p. 35) to Venter’s JCVI-syn3.0 and “a possible transition from biological evolution to biology by design” (p. 36). He names the driver: “our near-insatiable curiosity and our drive to better understand the world we live in and gain mastery over it” (p. 36).

(b) Could we, should we? (pp. 36–39). He sets up Malcolm’s line, then qualifies the moral reading straight away. “I’ve met remarkably few scientists and engineers who would consider themselves to be unethical or irresponsible” (p. 36). His diagnosis is instead that they are “so engaged with their work and the amazing things they believe it’ll lead to that they sometimes struggle to appreciate the broader context within which they operate” (p. 36).

Past episodes (the Industrial Revolution, the atomic bomb) were “just a rehearsal for what’s coming down the pike” (p. 37). InGen’s scientists are not fools. They build safeguards (the lysine contingency, all-female dinosaurs). But they are “so enamored with what they’ve achieved that they lack the ability to see beyond their own brilliance to what they might have missed” (p. 37). The lysine contingency is “about as useful as trying to starve someone by locking them in a grocery store” (p. 38).

This is “a salutary tale of scientists who are trying to be responsible—at least their version of ‘responsible’—but are tripped up by what they don’t know, and what they don’t care to find out” (p. 38). His positive definition of responsible science follows (p. 39, below). He admits “there are no easy guidelines or rules of thumb” (p. 39) and ends on a law-like point: “Complex systems behave in unpredictable ways” (p. 39).

(c) The Butterfly Effect (pp. 39–43). He covers Lorenz, Mandelbrot and Gleick. Malcolm’s pop-chaos includes “a lot of hokum,” for instance the idea that chaos theory can predict when chaos will occur (p. 41). The underlying point stands: “we cannot wield perfect control over complex technologies within a complex world” (p. 41). He then draws two constructive corollaries from chaos theory (p. 41; see concepts below). He tests realism with a real case: the Arkema organic-peroxide plant during Hurricane Harvey. Cascading small failures (“Overflowing toilets and snakes? Probably not,” p. 43) produced explosions and toxic releases. He then connects this to Perrow’s “normal accidents” (p. 43): “if Hammond had read his Perrow” (p. 43).

(d) Visions of power (pp. 43–45). Jurassic Park is about the power “to create and destroy life,” and also “the power to control others, to dominate them, and to win” (p. 43). The human creators “merely have delusions of power” over nature (p. 43). He identifies competing visions of power: investors, Gennaro as their proxy, and Hammond as “entertainer, charmer, and entrepreneur” (p. 44). He then rejects the simple morality-tale reading as “too simplistic a takeaway from the perspective of developing new technologies responsibly” (p. 44).

Power differentials are inevitable, often legitimate (“including the fiduciary responsibility of innovators to investors”), and “an essential driving force that prevents society from stagnating” (p. 44). “The challenge we face is not to abdicate power, but to develop ways of understanding and using it in ways that are socially responsible” (p. 44).

He lists several forms of power. Scientists wield it through knowledge, activists through methods and rhetoric, legislators through law, and “citizens collectively have considerable power over who does what and how” (p. 45).

Concepts and frameworks (his definitions)#

• De-extinction / “resurrection biology” (p. 32) and “reanimation” (Church’s term, p. 34). Three technical routes: selective breeding (aurochs), cloning, and DNA reconstruction/genetic engineering (pp. 32–34). He judges it “a stretch” to call Church’s approach de-extinction, since “what is being formed is a new species” (p. 34).
• Life as code. DNA is “a massively long and complex instruction set” (p. 33), “the biological software” (p. 33), with an Asian-elephant “starting source code” (p. 34). It needs biological “wetware” (p. 33). The read–recode–download capability (p. 35) and genetic “watermark[s]” that raise ownership questions (“who owns them?”, p. 36) are part of the same picture.
• Biology by design: “a possible transition from biological evolution to biology by design” (p. 36).
• The technological pull. “for some there is an almost irresistible pull toward using them, so much so that some would argue that not to use them would be verging on the irresponsible” (p. 35). He reports this view without endorsing it. The same idea returns as “the ‘can-do’ culture of Hammond’s Jurassic Park” (p. 37), and as “just like the scientists in Jurassic Park, we’re deeply caught up in what we can do as we learn to code and recode life” (p. 36).
• Could vs should. “where do the lines between ‘could’ and ‘should’ lie” (p. 37). The question gets harder as complexity rises: “the more complex the science and technology we begin to play with is, the more pressing this distinction between ‘could’ and ‘should’ becomes” (p. 39).
• Responsible science (his definition, p. 39). It is “about more than just having good intentions.” It also requires “the humility to recognize your limitations, and the willingness to listen to and work with others who bring different types of expertise” and knowledge (p. 39).
• Myopic/naïve science and “their version of ‘responsible’” (pp. 37–38). The failure is unintended consequences “arising from somewhat naïve and myopic science” (p. 37), and “the dangers of thinking you’re smart enough to have every eventuality covered” (p. 38). He adds an epistemic point: failure comes from what scientists “don’t know” and what they “don’t care to find out” (p. 38).
• Complexity / chaos / the Butterfly Effect (pp. 39–42). In a complex system “immeasurably small actions or events can profoundly affect what happens,” so accurate prediction is “well-nigh impossible” (p. 41). He presents chaos theory as challenging the belief that “if we have enough information, we can predict the outcomes of our actions” (p. 40).
• Bounded unpredictability / “predictable chaos” (pp. 40–41). “complex systems can lead to predictable chaos” and “even where chaotic unpredictability reigns, there are always limits to what the outcomes might be” (pp. 40–41). For futures thinking, “these boundaries become highly relevant in separating out plausible futures from sheer fantasy” (p. 41).
• Points of stability / squandered futures (p. 41). “within complex systems, there are points of stability.” So “there are some futures that are more likely to occur if we take the appropriate courses of action.” These are “futures that can be squandered if we don’t think ahead about our actions and their consequences” (p. 41). This is his bridge from unpredictability to agency.
• Cascades and normal accidents (pp. 42–43). “so often it’s these seemingly small events that help trigger larger and seemingly chaotic ones in complex systems” (p. 43). He uses Perrow’s theory “that, in any sufficiently complex technological system, unanticipated events are inevitable” (p. 43). The general claim: “technological innovation has the capacity to trigger events and outcomes within the complex social and environmental systems we live in that are hard to predict and manage” (p. 42).
• Visions of power / responsible use of power (pp. 43–45). Power “plays an oversized role in determining the trajectory of a new technology, along with any fallout that accompanies it” (p. 43). The norm he proposes is responsible power, not the abdication of power (p. 44). The goal: “a better future for as many people as we can, without sacrificing the health and well-being of communities and individuals along the way” (p. 45).

Themes#

• Risk. Risk here is mainly systemic and emergent: complexity, cascades, normal accidents, unintended consequences. Safeguards designed on narrow assumptions fail (lysine, all-female design). The value-based framing from Ch1 returns briefly: “Hammond’s dream” is itself at risk (p. 24, set up in Ch1).
• Responsible innovation. Responsible science means humility plus drawing in others’ expertise (p. 39). Responsible innovation, corporate social responsibility and “ethical entrepreneurs” are possible (p. 44), even though Jurassic Park shows none of them.
• Permissionless innovation. The term is not used. The adjacent ideas are the “can-do” culture (p. 37) and mega-entrepreneurs “dictating how new tech is used” (pp. 30–31).
• Hubris. The word is not used. The concept is everywhere, in several registers:
• “naïve human arrogance versus the triumphal dominance of chaotic, unpredictable nature” (p. 41), which is how he says the film frames it;
• “delusions of power” (p. 43);
• failing to “see beyond their own brilliance” (p. 37);
• “thinking you’re smart enough to have every eventuality covered” (p. 38);
• “in their cleverness, they’re not missing something important?” (p. 39).

His remedy is humility (p. 39). Note that he builds the hubris critique on epistemic limits (complexity, blind spots) rather than on vice. - Developers and their mindsets. - Scientists: well-intentioned, absorbed, locally responsible, blind to context (pp. 36–38). There is a footnote caveat that his portrayal is “enthusiastically short-sighted,” while the franchise later makes Henry Wu an evil scientist (p. 37, n. 13). - Entrepreneurs: Hammond is “charming, mega-rich, and, as it turns out, rather manipulative” (p. 28). He needs scientists’ “stamp of approval” to reassure investors (p. 28), which is an early nod to how scientific legitimacy gets used. More broadly, mega-entrepreneurs have outsized influence (pp. 30–31). But profit-driven innovation “has also created a lot of good” (p. 44). - Engineers/insiders: Nedry, the disgruntled systems engineer whose sabotage combines with a hurricane (p. 29). He treats this as part of the “catastrophic confluence of poorly understood technology, the ability of natural systems to adapt and evolve, unpredictable weather, and human foibles” (pp. 41–42). - Investors and lawyers: power through fiduciary claims (p. 44). - Governance and public engagement. Governance appears as distributed power rather than regulation as such. Law is one form of power among several, alongside knowledge, activism, capital and collective citizen power (p. 45). He warns of “two hundred years of environmental harm and human disease tied to technological innovation” (p. 44) while insisting power be harnessed, not renounced. - AI and intelligence. Minimal. Machine learning and data manipulation help reconstruct genomes (p. 33): AI as an accelerant inside biotechnology, which is convergence in action. Nedry’s control software is the single point of failure (p. 29), though he does not analyse it as an AI or software risk. - Convergence. Implicit throughout. Biology becomes digital (read, manipulate, recode, download, p. 35), with software metaphors for DNA (pp. 33–34) and machine learning for genome assembly (p. 33).

Analogies across technologies (Ch2)#

Chapter 14: Looking to the Future (pp. 287–291; Acknowledgments p. 292)#

Argument#

He writes this from the Isle of Arran, and it is a closing statement of stance. He briefly enjoys the low-tech island, where “happiness lies not in the latest technology, but in the more basic things of life” (p. 287), then rejects the nostalgia: “these dreams of a slower, more pleasant past are a sentimental illusion” (p. 287). Residents may see things differently (p. 287).

Three commitments follow:

1. An obligation to innovate, with equity in view. “emerging technologies, when developed and used responsibly, can and do improve lives in quite powerful ways” (pp. 287–288). “if we’re tempted to start renouncing technologies from a position of privilege, we risk denying too many people without the same privileges the chance to make their own decisions” (p. 288). “we have an obligation to explore new ways of using science and technology to improve the world we’re living in” (p. 288). The obligation carries “tremendous responsibilities”: ensuring benefit without harm, and “learning how to live responsibly in a world that, through our own drive to invent and to innovate, is constantly changing” (p. 288).
2. Responsibility cannot be delegated to experts. Leaving these questions to experts “is, in itself, an abdication of responsibility” (p. 288). Some questions “we cannot afford to leave solely to people like scientists, innovators, and politicians to answer” (p. 288). The aim is to “nudge them toward the future we want, rather than one that someone else decides for us” (p. 288).
3. “Don’t Panic.” Films risk leaving “a misplaced impression that we’re careering toward a hopelessly dystopian technological future, and there’s not a lot we can do about it” (p. 289). Yet “we shouldn’t be complacent—far from it” (p. 289). “I’m optimistic enough to believe that we have the collective ability to develop new technologies in ways that work for us, not against us” (p. 290). The two failure modes are panic, and becoming “so enamored by the tech itself that we become blind to its potential downsides” (p. 290).

He also restates the limits of his method. Films are useful “not because they are accurate or prescient, but precisely because they are not tethered to scientific accuracy” (p. 288), when “seasoned with feet-on-the-ground thinking” (p. 288). But “they cannot invent what’s yet to be discovered” (p. 289) and are “a poor guide to the technology itself” (p. 289). Science and technology cannot deliver the most important things on their own: “you can’t simply ‘science’ your way to them either” (p. 290).

The book is deliberately incomplete. Of the 70 WEF Top Ten Emerging Technologies he helped select, “only a handful” appear. There is no blockchain, quantum computing or self-driving cars, and the book will not “teach you how ‘deep learning’ works” (p. 291). “I set out to focus on how we think about technological innovation, society, and the future” (p. 291).

Concepts and frameworks#

• Obligation to innovate + privilege critique of renunciation (p. 288).
• Living responsibly in a self-transforming world (p. 288). Responsibility here means adapting to change we ourselves cause, not only preventing harm.
• Expert delegation as abdication (p. 288).
• “Don’t Panic” as a stance: neither dystopian fatalism nor complacency or tech-enchantment (pp. 289–290).
• Limits of techno-solutionism: “you can’t simply ‘science’ your way” to love and happiness, but technology can make them “that much easier to achieve” (p. 290).
• Thinking about innovation, not technology catalogues (p. 291). The focus is on how we think rather than on technical coverage.

Themes#

• Risk. Pitfalls are real: “a multitude of ways in which we can well and truly make a mess of things” (pp. 289–290). But the emphasis falls on agency, and on doing “more good than harm” (p. 290).
• Responsible innovation. Framed as obligation plus responsibility (p. 288).
• Permissionless innovation. Not named. The closest idea is his objection to a future “that someone else decides for us” (p. 288).
• Hubris. It appears as being “enamored by the tech itself” (p. 290).
• Governance and public engagement. Collective sense-making is the book’s closing argument: “we all need to be able to wrap our collective heads around what’s coming our way” (p. 288).
• AI. Only “deep learning” appears, as an example of what the book does not explain (p. 291).
• Convergence. Not developed here. He says “science and technology intersect and intertwine with society” (p. 291).
• Acknowledgments (p. 292). He mentions “the sequel to Films from the Future,” which signals an ongoing project.

Cross-chapter vocabulary check#

• Used: “risk innovation” (p. 23); “responsible innovation”/”Responsible Research and Innovation” (p. 22); “socially responsible innovation” (pp. 21, 24); “socially responsive” (pp. 24, 26); “convergence”/”converging technologies” (pp. 17–21); “unintended consequences” (pp. 21, 37); “could”/”should” (pp. 36–39); “arrogance”/”arrogant” (pp. 22, 26, 41); “humility” (pp. 25, 39); “power” (pp. 43–45); “normal accidents” (p. 43); “complex systems” (pp. 30, 39–43); “de-extinction”, “resurrection biology”, “biology by design” (pp. 30–36); “Don’t Panic” (pp. 289–290).
• Not used in these chapters: “permissionless innovation,” “hubris,” “precaution”/”precautionary,” “governance” (as a word), “manipulation” (only Hammond is “rather manipulative,” p. 28), “intelligence” as a concept beyond AI.

Digest: what FFTF-A contributes to a map of Maynard’s thinking#

These three chapters are the 2018 statement of Andrew Maynard’s working framework. Ch1 and Ch14 bracket the book and state its commitments outright; Ch2 shows the framework applied to one case. Together they give a baseline of positions, drawn from his own prose, from before generative AI dominated his writing.

1. Risk Innovation and risk as threat to value (most central, enduring). This is the conceptual core, and he says so: Risk Innovation is “where much of my current work lies” (p. 23), backed by a career in particle inhalation, nanomaterial and emerging-technology risk (p. 22). There are two moves. First, a mismatch argument: risk tools built since the Industrial Revolution for chemicals, materials, mining and pollution “belong to a different world” (p. 23). Hence the “new wine … old wineskins” image (p. 23). Second, a redefinition: risk means threats to “what is important to us,” including aspirations and intangibles such as dignity, belonging, identity, belief and “what it means to be human” (pp. 23–24). The second move is what later lets him treat effects on identity, cognition and being human as risk questions rather than merely ethical ones. It should carry heavy weight in any map.

2. Socially responsible (and responsive) innovation (central). Defined via RRI: everyone affected gets “a say,” and consequences are addressed early, because there may be no second chance (p. 22). He admits it is “fiendishly difficult” in practice (p. 22). Responsiveness, a word he repeats (pp. 24, 26), matters. The failure paradigm is Stratton, who “never bothered to ask anyone else” (p. 21).

3. A non-demonising account of developers (central, distinctive). Scientists “care deeply” (p. 17), are rarely self-consciously unethical (p. 36), and “they’re not fools” (p. 37). They fail through absorption, narrow (“their version of ‘responsible’”) safeguards, and not seeking out what they “don’t care to find out” (p. 38). His hubris critique is epistemic and structural rather than moral. The remedy is humility plus other people’s expertise (p. 39). The same even-handedness applies to power. The “corporate greed” morality tale is “too simplistic” (p. 44), and power should be used responsibly, not abdicated (p. 44). This combination of criticism of entrepreneurial overreach (“mega-entrepreneurs,” pp. 30–31; “entrepreneurial arrogance,” p. 26) with a refusal to villainise is a signature of his thinking.

4. Complexity and bounded unpredictability (central method). From Lorenz, Mandelbrot, Perrow and the Arkema cascade he draws three points: perfect control is impossible (p. 41); outcomes are nonetheless bounded, which lets us separate “plausible futures from sheer fantasy” (p. 41); and “points of stability” mean good futures are reachable but “can be squandered” (p. 41). This turns complexity from a counsel of despair into a basis for foresight and steering. The Arkema example is also his clearest cross-technology move: a real chemical-plant cascade used to test a fictional biotechnology failure.

5. Convergence (central in 2018; AI is one strand). Convergence across physical, biological and cyber technologies is where both “transformative power” and “the greatest potential pitfalls” lie (p. 20). The capability runs ahead of understanding: “little if any idea what might go wrong” (p. 21). In Ch2 convergence is concrete: DNA as “biological software” (p. 33), read–recode–download (p. 35), machine learning assembling genomes (p. 33).

6. Distributed responsibility and public engagement (central, enduring). Everyone is “qualified” to judge consequences for those they care about (p. 26). The overlooked are “especially” needed (p. 18). Leaving it to experts is “an abdication of responsibility” (p. 288). Art and film serve as “a common point of focus” across ideological divides (p. 25), alongside technical expertise, law and policy, not instead of them. Governance is pictured as distributed power (knowledge, capital, law, activism, citizens, p. 45), with regulation expected to avoid “unnecessary roadblocks” (p. 25).

7. Responsible optimism: obligation to innovate, “Don’t Panic” (central tonal stance). He sees renouncing technology from privilege as an equity harm (p. 288) and asserts “an obligation to explore” (p. 288). Both panic and tech-enchantment are errors (p. 290). Equity runs as a steady undertone (pp. 15, 18, 20, 45).

8. AI (peripheral here, but a useful baseline). In 2018 AI appears as HAL’s self-preservation (p. 15), a claim that current AI is already “faster and smarter than any human” (p. 17), one strand of convergence (pp. 19–20), and an open question between “a better place” and “the end of humanity as we know it” (pp. 20–21). He takes no position on AI risk here. This marks where his AI thinking starts before later work.

Absences worth recording: “permissionless innovation,” “hubris,” “precaution” and “governance” as terms do not appear in these chapters. The ideas of hubris and unconsulted innovation are pervasive; the vocabulary is “arrogance,” “can-do,” “could/should.”

Relative weight: risk innovation and value-based risk, and socially responsible innovation, are foundational. Complexity-and-steering, the non-demonising view of innovators, power realism and distributed public responsibility are core, recurring analytic habits. Convergence is a major 2018 organising theme. De-extinction and biology-by-design are case material that illustrate the framework. AI is marginal in these chapters.