T1. What risk is, for Andrew Maynard#

Thread. Risk as a threat to value (not only to health or environment); hazard versus risk; risk innovation; the “risk landscape”; risk thinking as the opposite of fear-mongering; orphan risks as one expression among many; how risks should be identified, framed and navigated.

Evidence base. The batch digests and notes (B01–B32), the Films from the Future notes (FFTF-A to F), concept-index.md, timeline.md, and a direct reading of about forty posts and the relevant book pages. Only Andrew Maynard’s own prose is evidence. Provenance cautions:

• 2026-07-16 orphan-risks-frontier-ai-maynard. His prose, rewritten from a Fable 5 draft. He credited Fable with applying risk innovation to frontier AI “in a way that hadn’t previously occurred to me” (2026-07-04 just-how-good-is-anthropics-fable-as-a-research-assistant). Threat to value, orphan risks, the Planner and the Garbee lesson are his; the four “filters”, “safety differential”, register and aperture log are apparatus he adopted, not originated.
• 2026-09-24 being-an-academic-in-an-age-of-ai. His lecture, drafted into prose by Claude and line-edited by him: ideas secure, wording less so.
• 2023-11-21 ai-and-risk-innovation. His framing only. 2019-08-13 responsible-innovation is co-written. Modem Futura posts are excluded at the user’s request. Pre-2019 dates are approximate.

Where I interpret rather than report, I say so.

1. The position in brief#

For Maynard, risk is not in the first instance a number attached to a hazard. It is a threat to something that someone values. That something may be health, environmental integrity or money, but it may equally be dignity, identity, belief, agency, livelihood, trust, hope, or a future someone aspires to. He first set this out publicly in January 2016 as the foundation of risk innovation, “parallel innovation in how we think and act on risk” (2016-01-11 thinking-innovatively-about-the-risks-of-tech-innovation). He has held it, with its definition essentially unchanged, for a decade. It is the most portable idea in his work. It is what lets him treat AI’s effects on identity, cognition and belief as risk questions rather than as ethics alone.

Three commitments sit alongside it and constrain it:

1. Professional risk science. He came to risk as an occupational aerosol and nanomaterial scientist, and he keeps that discipline’s grammar: hazard is not risk, exposure converts one into the other, consequences matter as much as probabilities, and evidence is weighed, not cherry-picked.
2. Risk is social. What counts as harm, what is “safe”, and what is acceptable are set by people, not by engineering. “There is no such thing as absolute safety” (2024-06-20 ilya-sutskevers-safe-superintelligence-rethink). Who decides is therefore always part of the question.
3. Risk is symmetric. Losing future value counts too, so not developing a technology is itself a risk, and so is a badly designed precaution. This is why his risk thinking never collapses into a default “no”.

From these follows his view of what risk thinking is for. It is a navigational discipline, the means by which benefits are realised: “you can only begin to realize the benefits of a technology if you understand what can possibly go wrong” (2026-09-24, lecture). It is explicitly not fear-mongering. He polices both edges: against Hollywood catastrophe and “make-believe” treated as plausible, and against complacency, dismissal and the equation of risk talk with pessimism.

Orphan risks, named in 2018, are one tool inside this frame. They are the hard-to-quantify, easy-to-ignore threats to value that conventional approaches sideline. The concept is recurring and useful but never the organising centre. Its 2026 frontier-AI application is the latest in a long line of applications of the threat-to-value idea.

2. Formation: a risk scientist who lost patience with risk#

His framing grows out of a career he describes directly in the book. He spent thirteen years in workplace dust and aerosol research at the UK Health and Safety Executive and NIOSH, had a grandfather who died of coal miner’s pneumoconiosis, worked on nanomaterial health risk, co-chaired the NNI’s environmental health and safety committee, and advised the Project on Emerging Nanotechnologies (FFTF pp.118–122, 215). From this he concludes: “I have less and less patience for how many people tend to think about risk” (FFTF p.22).

Lasting convictions come from this material. Technical sophistication does not produce safety: “there is relatively little correlation between the sophistication of the technology and the safety of the environment in which it’s used” (FFTF p.121). Harm lands first on the “first tier” of workers (FFTF p.121). Uncertainty is not neutral: “This was an uncertainty that suited the mine owners” (FFTF p.120). And risk outlives attention: “new technologies all too easily slip under the radar of critical public evaluation, simply because few people know what questions they should be asking” (2016-02-01 we-dont-talk-much-about-nanotechnology-risks-anymore).

Two points of interpretation matter for everything that follows. First, his early risk writing frames harm as a matter of justice: who bears it, and who profits from uncertainty. Second, his impatience with conventional risk thinking comes from inside the field. He does not reject risk science. He argues that it is necessary and no longer sufficient.

3. The core move: risk as a threat to value#

3.1 The definition#

The founding statement (2016-01-11 thinking-innovatively) borrows from innovation itself. Innovation creates value someone will pay for, so risk can be understood as a threat to value. The move is to count “not just in the ways value is usually thought of when assessing risk, such as health, the environment or financial gain/loss”, but “less tangible but equally important measures of value — well-being, environmental sustainability, deeply held beliefs, even a sense of cultural or personal identity”. The list soon grows: self-worth, culture, sense of security, equity, “sacrosanct beliefs” (2016-03-02 how-risky-are-the-world-economic-forums-top-10…).

The book states it most fully. Films show “threats to dignity, belonging, identity, belief, even what it means to be human”. Risk concerns “what is so important to us that our lives are diminished if it’s denied us, or taken from us”. It is “something we have and can’t face losing, or something we aspire to and cannot bear to lose sight of” (FFTF pp.23–24).

The compact definition he repeats from 2018 onward: risk is “a threat to something of importance to an individual, a community, or a business organization” (2018-12-13 tech-startups-orphan-risks; reposted unchanged in 2023-11-15 navigating-orphan-risks).

Later lists keep widening the object: hope, justice and scientific integrity (2020-07-30 life-on-mars…thinking-differently-about-risk); “equity, agency, and a sense of self” (2020-10-15 the-ethics-of-advanced-brain-machine-interfaces); “deeply held beliefs, and even self-identity” as values frontier AI threatens (2023-07-12 regulating-frontier-ai-models).

3.2 Aspiration and future value#

From the start the definition covers what people hope for as well as what they have. In the book, characters risk “either losing something of great importance to them, or being unable to gain something that they aspire to” (FFTF p.24). In 2023 risk is “not just about protecting what we have, but protecting pathways to futures we desire” (2023-11-26). In 2024 it becomes “a balance between maintaining existing value, and enabling the creation of future value” (2024-08-25 advanced-technology-transitions-model). This forward-looking element is what makes the definition symmetric (section 7).

3.3 Value, not values#

He distinguishes the two in 2016, parenthetically: “tangible value (not to be confused with ‘values’)” (2016-01-11). He formalises the distinction in the AI period: - “values are not the same as value”, and the approach is “agnostic to particular worldviews, ideologies, or ethics” (2023-11-21, his framing); - “value” means “the worth of something to someone”, something “that can be lost or gained” (2024-08-25); - values “are important, but the former is more effectively operationalized in policy and decision-making” (2024-12-17 navigating-the-challenges-and-opportunities-of-advanced-biopreservation-technologies).

His stated reason is practical. “Value” lets organisations and communities name what matters to them in their own terms, without first agreeing on right and wrong.

3.4 Reciprocity: risk runs through relationships#

The frame looks outward as well as inward. From 2016 the constituents of risk are those “who have something of tangible value … that is potentially threatened, and that they are willing to invest in protecting”. This is a risk “market” that includes developers but is not limited to them (2016-01-11). The failure mode he warns of is “inadvertently threatening what people are prepared to fight for” (2016-01-11).

The 2018 version names “the reciprocal dangers of threatening what is important to others through what they do” (2018-12-13). By 2024 it is operationalised as internal threats versus reciprocal threats: an organisation threatens its stakeholders’ value, and that in turn threatens its own (2024-12-17). In 2026 it is compressed into “your risk is my risk” (2026-07-16, Fable-assisted; the idea itself is his and goes back to 2016).

The same logic explains resistance to technology. The mill workers in The Man in the White Suit are “shrewd enough to see how change supports or threatens what they value, and they fight to protect this value” (FFTF p.225). SpaceX’s “nontechnical hurdles” come down to whether society “grants” it freedom, which it may not “if enough people feel SpaceX is threatening what they value” (2017-04-10 dear-elon-musk…). Moral panics are not to be mocked, because “threats to what’s important to people can lead to responses that may seem irrational on the surface, but are usually more complicated underneath” (2025-06-01 vibe-coding-moral-panic).

3.5 Extension, not replacement#

He is consistent that the value frame “extends conventional thinking rather than replacing it — health, wealth and the environment all fit comfortably into the ‘value’ bucket” (2018-12-13). In 2026 he puts it this way: “This does not abandon the idea of risk as involving the probability of harm. Rather, it widens what counts as harm” (2026-07-16).

3.6 How firmly he holds it#

Very firmly. The definition has barely changed between 2016 and 2026. He reposts the 2018 statement unchanged in 2023. He re-endorses the 2019 Neuralink risk landscape as one that “still stands up reasonably well” (2024-03-21 elon-musks-neuralink-plays-mind-games). He re-endorses his 2021 AI-and-risk talk as “perhaps more relevant than ever in an age of generative AI” (2023-08-08 thinking-differently-about-ai-and-risk). The one hedge he offers is epistemic, not a retreat: “Approaching risk as a threat to value is, admittedly, a somewhat subjective way of thinking about risk” (2018-12-13).

What has changed is the frame’s reach: - startups and investors (2016–2021); - his definition of AI catastrophe (2023-05-31); - the axis of his transitions model (2024-08-25); - the language of care in teaching (2025-05-18); - frontier-AI governance (2026-07-16).

4. The professional grammar: hazard, exposure, consequence, evidence#

4.1 Hazard is not risk#

The earliest posts argue against hazard-based alarm. Quantum-dot TVs contain cadmium and engineered nanoparticles, “Yet taken in isolation they are misleading”. Once exposure is assessed across use, manufacture and disposal, and net cadmium emissions are counted, the case shows “the dangers of jumping to conclusions over risks without seeing the full picture” (2015-01-10 are-quantum-dot-tvs…). The two Vantablacks make the same point. The original was carbon-nanotube-based and space-bound: “It wasn’t nontoxic, but the risk of exposure was minuscule”. The sprayable version raises “serious risk questions” because people might touch it, inhale it or swallow it (2016-02-01).

The clearest statement is in the 2019 algorithms-as-chemicals essay: “conflating hazard with risk ultimately doesn’t help anyone, as everything has the potential to cause harm buried somewhere within it”. His images are bleach under the sink and a grizzly bear seen at a distance versus met face to face. “No exposure means no risk, even if a chemical is potentially deadly” (2019-03-05 should-we-be-treating-algorithms-the-same-way-we-treat-hazardous-chemicals).

4.2 Carrying the grammar to algorithms (2019)#

In that essay he transfers five concepts from chemical risk assessment (hazard versus risk; the events that turn potential harm into actual harm; consequences; exposure–response; checks and balances on evidence), insisting the transfer is structural: “an algorithm is not a chemical”. He coins algorithmic exposure: “Anyone who is potentially impacted by the deployment of an algorithm can be thought of as being exposed to it in some way”. Consequences count because “the type of harm is as important as the probability of harm occurring”, and the harms include livelihood, “liberty, dignity, and self-respect”. Here the value frame appears inside the technical one. Two later constants are fixed here too: “there’s no such thing as zero risk”, and “knee-jerk reactions to seemingly-startling results rarely result in socially beneficial outcomes”.

4.3 First principles, applied to AI (2023)#

His fullest analytic treatment is 2023-11-26 everything-youve-heard-about-ai-risk-is-wrong. He starts from the textbook definition, risk as “the probability of harm occurring from an action, process, or situation”, and unpacks five elements: - Cause and effect. “no cause, no risk”. Hazards and speculation (“AGI going rogue for instance”) are not risks: “if there is no causal pathway between hazard or speculation and outcomes, there is no risk”. - Magnitude. Deciding what is negligible or substantial “means that risk is ultimately a social construct as it reflects broad societal values and norms”. - Harm. Harm understood as “diminishing or destroying something of value or worth”. Once it extends to wellbeing, identity and beliefs, causal links become “increasingly hard to define”. - Time. Chronic and delayed effects blur cause and effect even in conventional risk science. - Perception. “an understanding of risk cannot be separated from an understanding of human behavior and societal dynamics”.

His verdict is that AI takes each element “to a whole new level”. We know almost nothing about the relevant causes, effects, harms (including those of not developing AI), timescales, or whether regulation could itself be a risk. The result is an “understanding-vacuum” that fills with “dogmatic overconfidence”. The remedy he gives is procedural: humility, dropping egos, transdisciplinary collaboration and agility. He also admits his own title was “a little hubristic”.

A next-day addendum tests risk = Fn(hazard, exposure) on AI. He explains he had avoided the paradigm because it can get “gnarly” (thresholds, hormesis, low-dose effects), and because “I also didn’t want to fall into the trap of implying that zero exposure — as in no AI — is a default risk management strategy”. He then sketches what AI hazard might mean (“as subtle as influencing human behavior”) and what exposure might mean (“as intangible as hints of ideas encountered over hours of social media use”). He concludes by noting “the lack of even the beginnings of a framework” for AI hazard, exposure and transfer function.

My reading: this is the first place where his chemical training reaches toward the cognitive exposure that becomes central to his 2026 work on epistemic vigilance.

4.4 Behaviour over labels; process over outcome#

Two further professional commitments shape how he identifies risk: - Behaviour over labels. Governance built on definitional categories is fragile, because “nature doesn’t care what we call a material, it just cares about how it behaves” (2022-02-10 are-we-asking-the-right-standards-questions…, restating his 2010–11 “sophisticated materials” work). - Process over outcome. Responsibility is judged by process. Selling graphene masks without answering four basic questions is “irresponsible innovation on a grand scale — even if the risks turn out to be negligible”. The questions are: can it get into the body; can it cause harm; how; and how much is needed (2021-03-28 how-safe-are-graphene-based-face-masks).

4.5 How the two definitions fit (interpretation)#

He works with two definitions of risk and moves between them: 1. Probability of harm, the risk scientist’s definition, used in 2019, 2023 and 2024. 2. Threat to value, the risk-innovation definition.

He says the second extends the first, and in 2026 he says it “does not abandon” probability. In practice they do different jobs: - The first disciplines claims about whether harm will occur: exposure, causation, evidence. - The second widens what counts as harm, and whose harm counts.

The frame “risk as a threat to value” is not a probabilistic definition. It is a definition of the object of risk. His writing does not say how the two are combined in a single judgement (see section 12).

5. Risk is social: harm, safety, acceptability and who decides#

From 2016 onward he treats the boundaries of risk as socially drawn:

• Who defines risk. Responding to risk “depends as much on who considers what a risk (and to whom) as it does the capabilities of emerging technologies to do harm” (2016-03-31 considering-ethics-now-before-radically-new-brain-technologies…). “numeric logic is often trumped by what we intuitively think and feel is important” (2016-03-12 itll-take-more-than-tech-for-elon-musk…). “It’s easy to make risk decisions when you’re not the one who has to suffer the consequences” (2020-07-30).
• Safety is social. His sharpest statement is aimed at an AI-safety venture. “There is no such thing as absolute safety”. Harm is “a social construct, not a technological one”. “zero risk — the corollary of absolute safety, is only possible in the absence of change”. Risk is “the operationalization of safety, it’s never zero, acceptable risk is ultimately governed by what people agree on”. The question never asked is “who decides what ‘safe’ means” (2024-06-20). Even bridge safety, he notes, is set by norms and standards.
• Acceptability is perceived. Safety “is deeply grounded in a perception of what is considered to be acceptably safe — which is highly subjective”, so humanoid robots face societal more than technical barriers (2024-08-07 are-humanoid-robots-really-the-future).
• Publics can judge. People “don’t need to understand the inner workings of AI” to judge how it might “threaten what’s important to them” (2023-05-15 erik-schmidt-ai-regulation).

He also treats expert risk perception as socially shaped. Crowdsourced WEF rankings reflect a “risk perception zeitgeist”, driven “more by headlines than foresight”, and need foresight to “augment the wisdom of the expert crowd” (2024-01-14 wef-global-technology-risk-trends). In 2025 he adds that such opinions “tend to regress to the mean”. That helps avoid over-focusing on speculative risks but devalues poorly understood ones. Experts ranking AI low “do not grasp how disruptive the technology may turn out to be”. Disruptions will be “‘I told you so’ moments” for insiders and “blindsides” for everyone else (2025-01-19 wef-global-risks-2025).

In the book this social view extends to perception and complacency: “a risk not experienced is a risk not worth worrying about” (FFTF p.256), and the “wow” to “meh” habituation (FFTF pp.284–286).

My reading: his “social construct” claim is about the evaluation of risk (what counts as harm, what is acceptable), not about whether hazards are real. He holds both views at once. That is how he can say “nature doesn’t care what we call a material” and also “risk is ultimately a social construct” without sensing a contradiction.

6. Risk innovation and the risk landscape#

6.1 The mismatch argument#

Risk innovation begins from a mismatch between our risk tools and our technologies. Regulations “are inevitably built around previous technologies”, and shoehorning new ones in “obscures potential pitfalls” (2016-01-11). Tools built since the Industrial Revolution “belong to a different world than the one we’re now creating”; we risk trying to “squeeze the new wine of technological innovation into the old wineskins of conventional risk thinking” (FFTF p.23). By 2020 “our outmoded ideas about risk actually become a risk in themselves” (2020-11-05 risk-innovation-and-the-future).

6.2 Risk innovation as innovation#

The analogy with product innovation is deliberate. “Rather than framing risk as a barrier to progress, risk innovation transforms it into a way of supporting beneficial and sustainable progress” (2016-01-11). It “seeks to transform creative ideas around how we think about risk into frameworks and processes that people want to use” (2020-07-30).

The institutional line runs from Michigan entrepreneurship students (2013) to the ASU Risk Innovation Lab (2015), the Accelerator (2017), the Nexus (2019) and the end of seed funding (2020) (2020-11-05). That explains its early audience (startups and investors) and its tone: risk as a condition of success, not a moral lecture. The lesson he draws from that work: “if you want a fast-moving organization to attend to a risk, you do not hand it a compliance duty; you show it a threat to something it values” (2026-07-16).

6.3 The “risk landscape”#

“Risk landscape” is his recurring spatial metaphor. It appears in about twenty-five posts. Its meaning is fairly precise: - Terrain between an idea and its realisation. The landscape “lies between new ideas and their successful implementation” (2018-12-13). In 2024 it lies “between where an enterprise currently is, and where it wants to end up” (2024-12-17). - Co-produced. Technologies “both face and help to form” their risk landscapes (2016-01-11). - Mappable but subjective. The 2019 Neuralink map charts orphan risks against areas of value for the enterprise, investors, customers and communities, looking for clusters that “converge” into “truly blindsiding impacts”. He calls it “a subjective process, and one that needs to be treated with some caution” (2019-11-01). - Crowded, shifting and navigated. Companies should keep “iterating frequently as the landscape shifts and changes” (2019-11-01). AI presents “an increasingly complex AI risk landscape that’s going to take massive collaborative efforts and a good dose of humility to understand and navigate” (2023-11-26).

My reading: the metaphor carries his whole stance. Risks are features of a terrain you must cross to reach something you value. They are not verdicts that stop you. The operative verb is always “navigate”, not “eliminate” or “avoid”.

6.4 Why risk rather than ethics#

In the AI period he uses risk innovation to argue against an ethics-led framing. In 2021 he showed bibliometrically that AI ethics was booming while research on AI’s own risks was thin; most “AI and risk” papers “focus on how to use AI in more effectively assessing and managing non-AI risks” (2021-08-03 we-need-to-get-more-innovative…). In April 2023 he calls for “moving away from an ethics framing to one around risk and socially responsible/beneficial innovation”. His reason: ethics helps “parse out what is considered right and wrong”, but does not “on their own provide a practical framework for achieving safe and beneficial technologies”. He places risk innovation, “aimed at supporting agile decisions around intangible risks”, in the governance lineage of agile and anticipatory governance and soft law (2023-04-04 what-are-the-alternatives-to-calling). His 2019 Neuralink work frames the same move at the level of the enterprise: “Rather than critique brain-machine interface technology on ethical grounds, we asked what the orphan risks landscape might look like” (2019-11-01).

6.5 Orphan risks: one tool in the kit#

Orphan risks are best understood as the operational edge of the threat-to-value frame. Their definition has shifted slightly over time, while their place in the frame has stayed the same:

• 2018, when he named them “a new” concept. Risks “perceived as being too ill-defined, too complex, or too irrelevant to be worth paying attention to” that can “derail entire enterprises”. They are “‘known knowns’ if you’re looking in the right place”, mostly social, and rarely governed by law (2018-12-13).
• 2019–2021, a method. A list of eighteen, mapped against areas of value. Orphan risks are “hard to quantify threats to value that often slip between the cracks of conventional risk approaches” (2020-10-15). The three questions: “What’s important to your enterprise, your investors, your customers, and the communities you touch? What orphan risks threaten these things of importance?”, then what small steps to take (2021-09-07 should-we-be-worried-about-elon-musks-tesla-bot).
• 2023, re-aimed at AI developers, after Galactica’s failure, a case of companies “technologically creative yet socially naive” (2023-11-15).
• 2024, tested beyond startups with an NSF biopreservation consortium: “risks that are often overlooked because they’re messy, subjective, and hard to deal with” (2024-12-17).
• 2026, from noticing to owning. Orphan risks become risks “for which no agreed-on tools, standards or mitigations exist, which no one is accountable for in practice”, known but unowned. The paper argues that frontier labs’ probability-of-severe-harm definitions filter out gradual, hard-to-measure harms such as manipulation and the erosion of trust and epistemic agency. Those frameworks are “working as designed. It’s just that the design itself may be flawed” (2026-07-16, Fable-assisted apparatus).

Throughout, orphan risks derive from threat-to-value. They appear on sixteen posts in eleven years (timeline thread counts), and are absent from his 2024–26 cognition essays and the September 2026 lecture. Their 2026 prominence is one application in a sequence.

7. Symmetric risk: the risks of not innovating, and of caution itself#

Because value includes future value, he counts forgone benefits as risks from the first. The value frame “opens the door to considering the potential risks of not developing a technology” (2016-03-02). “there’s always a risk of not developing or using a product” (2019-03-05). Catastrophe as loss of value must include “the potential loss of solutions to pressing challenges” (2023-05-31 existential-risks-of-ai). Missed opportunities are a threat quadrant in his transitions model (2024-08-25). And inertia is a risk: “sacrificing what could be on the altar of a blind devotion to what is” (2025-03-30 reimagining-education-in-an-age-of-ai).

The same symmetry applies to precaution. In the book, the Yellowstone thought experiment shows precautionary action has its own victims: “Pushing for action based on available evidence always comes with consequences” (FFTF p.243). He also refuses to dismiss inaction: “we cannot afford to dismiss the possibility that inaction in the present may lead to catastrophic failures in the future” (FFTF p.241). He calls the precautionary principle “often misunderstood”. Yet he endorses the UNESCO COMEST formulation (plausible but uncertain, morally unacceptable harm; a proportionate response; a participatory process) as “a sound philosophy for addressing complex, uncertain, and potentially catastrophic risks before it’s too late” (2020-07-30). He scales caution to irreversibility: with gene drives “we don’t have the luxury of rebooting when things go wrong” (2016-01-20 three-ways-synthetic-biology…).

In the AI period this symmetry shows in several positions: - He declines the 2023 pause letter while believing in “a risk of potentially existential proportions … (I do)”, and argues “the biggest risk is not taking action” (2023-04-04). - He rejects “zero exposure — as in no AI” as a default (2023-11-26). - He defines the AI risk space as “the risks of going too fast, the risks of not going fast enough, or the risks of simply assuming there are no risks” (2023-11-26). - By 2026 he adopts, as a flagged working assumption, “We can’t pause it” (2026-09-24). - The exception is a narrow, conditional openness to “pausing — or even rethinking” emotion-exploiting companion chatbots (2024-10-27 personal-ai-chatbots-and-stochastic-agency).

8. Risk thinking is not fear-mongering#

This is one of his most constant self-descriptions. It has three parts.

8.1 Against spectacle and make-believe#

From 2016 the real world is “less ‘zombie apocalypse’ and more ‘teens troll supercomputer; teach it bad habits’” (2016-03-02). The book makes plausibility his main filter: - Speculation becomes dangerous “when make-believe is treated as plausible reality”, as in the ITS bombings of nanotechnologists fed by gray-goo fears. It also works through policy, advocacy and investment, including the “tragedy” of forgoing beneficial technologies (FFTF pp.195, 205–206). - Occam’s Razor places gray goo and superintelligence on “a house-of-cards stack of assumptions”. Funding them over evidence-based materials harms is “more an act of faith than of reason”, though the probability is “not a zero probability” (FFTF p.281). - AI risks are “far more mundane–but no less serious for this” (2020-11-12 is-artificial-intelligence-going-to-kill-us-all). - AI risk is a whole “landscape”, not a ranking centred on extinction (2018-05-12 10-potential-risks-of-artificial-intelligence…, reaffirmed 2026-09-15).

Extinction is “both too narrow and absolute a framing, and too human-centric”. Catastrophe is better understood as events where “large numbers of people risk losing something that is deeply valuable to them” (2023-05-31).

8.2 Against dismissal and complacency#

He is equally hard on the other edge. The 2016 post names “the greatest risk”: that “either in our enthusiasm for developing these technologies, or our Hollywood-inspired fears of potential consequences, we lose sight of the value” of good technology (2016-03-02). “Don’t Panic” rejects panic and also becoming “so enamored by the tech itself that we become blind to its potential downsides” (FFTF p.290). The concerns behind the pause letter “are not typically based on science fiction speculation or cynical fear mongering” (2023-04-18 universities-need-to-be-investing). Eye-rolling at existential risk is “true at times”, but “simply ignoring the possibility of potentially catastrophic events … is in itself a risky strategy” (2024-06-23 existential-risk-jay-baruchel).

8.3 Talking about risk is how benefits are realised#

His late statements make the point directly. Risk Bites aims “not to stoke fears (not my style)”, and “it never ceases to amaze me how many people equate talking about risk with fear mongering. And yet, it’s pretty much impossible to manage risks if you don’t talk about them”. The two failure modes are refusing to talk about risk, or “freaking out while ignoring people and institutions who know a thing or two about risk”. Existential risk should be handled “without running around like headless chickens” (2026-09-15 will-ai-really-kill-us-all). In 2026-05-10 do-not-do-this-with-ai, “putting the safety message first is necessary — because while the benefits of a powerful technology are often self-evident, the risks are not”. Talking about AI risk “gets you branded as a technology-pessimist, or even a Luddite”, yet “understanding and navigating risk is absolutely essential to reaping the long-term benefits of any powerful technology”.

My reading: for Maynard, fear and dismissal are the same error. Both replace evidence-weighted, plausibility-checked attention to what people value with instinct. His own term for the alternative is “navigation”. Section 12 notes that his tone darkens in 2026 (“one of the scariest things I’ve ever seen”, 2026-09-24), while this stance stays formally the same.

9. How risks should be identified, framed and navigated: his method, assembled#

He never sets out a single method. Taken together, his writing implies the following sequence. The sources are his; the ordering is mine.

1. Start from value, not hazard. Ask what matters to the enterprise and to the investors, customers and communities it touches (2019-11-01; 2021-09-07). Ask what the people affected “have and can’t face losing” or “aspire to” (FFTF pp.23–24). “Community” includes people disadvantaged by not using a product (2023-11-21, his framing).
2. Look deliberately for what conventional tools miss. These are the hard-to-quantify, easy-to-ignore threats: orphan risks. AI risks may “blindside us, in part because we’re not thinking creatively enough about how an AI might threaten what’s important to us” (FFTF p.174). He treats risks that are hard to see as the priority: “the ones that are hard to see yet impact what is most valuable to us” (2026-05-10).
3. Map the landscape; look for clusters and trajectories. - Clusters of risks can converge into blindsides (2019-11-01). - Pathways link near- and far-term threats and opportunities, with mechanisms that open or close them (2024-08-25). - Cause and effect can be non-linear (S-curve, exponential, hysteresis, jagged, chaotic), and “effect” means “what we consider to be of value — or what we care for” (2025-05-18 exploring-ai-through-cause-and-effect). - Risk is judged on trajectory, “what might be possible given current trends” (2025-07-06 ai-risk-motive-means-and-opportunity).
4. Discipline claims with risk science. Separate hazard from risk and find the exposure pathway. Weigh the type of harm as well as its probability. Rely on the weight of evidence. Test futures for plausibility (2019-03-05; 2023-11-26; FFTF pp.168–171).
5. Weigh risks against risks. Count forgone value, the costs of precaution, and regulation itself (2016-03-02; 2023-11-26).
6. Decide acceptability socially, with the people affected (2024-06-20; 2023-05-15; FFTF p.227).
7. Treat irreversibility and complexity as reasons for more care. Neglecting “a large portfolio” of risks invites “serious and irreversible failures” (2025-07-23 americas-ai-action-plan).
8. Act in small, iterated steps and build a mindset. The Planner asks for a handful of small actions, repeated (2021-09-07; 2023-11-21). Risk innovation is a “mindset” (2020-11-05).
9. Communicate plainly, and don’t rely on warnings or literacy. From “decades” in “risk assessment, management and communication”, he is “deeply skeptical” that telling people to “be careful” works. Literacy classes “risk becoming performative”. Trust, dialogue and safe spaces work better (2025-11-09 universities-chatgpt-mental-health). “nothing in what we know about risk behavior and risk communication suggests” literacy alone will work (2026-05-10).
10. Make choices about scope visible. In 2026 he endorses disclosure of which risks a firm chose not to manage, and why (2026-07-16, adopted apparatus).

10. Development over time#

Constants. Risk as a threat to value, with its definition unchanged. Hazard is not risk. No zero risk. The risks of not innovating. Plausibility over imaginability. Navigation rather than prohibition. Publics as legitimate judges of what they value. Suspicion of both hype and doom.

Shifts. 1. Audience. Entrepreneurs and investors (2016–2021), then AI developers and policymakers (2023), then institutions and the general public (2025–26), with a governance and disclosure turn (2026-07-16). 2. Object. Material and bodily harm (nano, implants), then social and relational harm, then cognitive and epistemic harm. The 2023 addendum’s “hints of ideas” exposure grows into epistemic agency as a named blindside (2026-07-16). 3. Emphasis on communication. From explaining risk (Risk Bites) to “safety message first” and doubts about literacy (2025–26). 4. Tone. From “Don’t Panic” (2018) to “one of the scariest things I’ve ever seen” (2026-09-24), without abandoning the anti-alarmist stance. 5. Tails and exponentials. In 2018 exponential extrapolation is a fallacy (FFTF pp.199–202). In 2025 exponential blindness is a human failing, and tails are worth planning for “on the off chance” (2025-04-06 responsible-innovation-and-ai-acceleration; see concept index 5.3). His risk frame now makes more room for low-probability, high-consequence reasoning than the 2018 Occam’s Razor passage did.

11. Connections to his other threads#

• Responsible innovation. Risk innovation is its operational partner, the part that supplies a “practical framework” where ethics alone does not (2023-04-04). As his confidence in responsible innovation falls (2024–25), the value frame is what he keeps.
• Being human. “what it means to be human” is named as a value at risk from the start (FFTF p.23). By 2026 the “who we are” domain is where he places AI’s most distinctive risks (2026-05-21 magnifica-humanitas-and-being-human). My reading: in risk terms, the being-human thread is an account of which values are most exposed.
• Manipulation, cognition and formation. Heuristic manipulation appears in the 2018 list, cognitive “exposure” in 2023. The cognitive Trojan horse is argued on risk-analytic grounds, with a small chance enough to warrant research (2026-01-10 is-ai-a-cognitive-trojan-horse). Emotional reliance and epistemic agency are the blindsides he names in 2026 (2026-07-16).
• Complexity and irreversibility. Non-linear failure (2020-11-05) and his six cause–effect models (2025-05-18) justify attention to small, overlooked risks.
• Advanced technology transitions. Threat to value is the axis of his transitions model (2024-08-25). “Novel theories, models, and approaches to risk” is a research domain of the field (2023-09-25 building-a-better-futures-tough).
• Who decides, justice and governance. Risk turns on who bears harm and who sets acceptability. He prefers agile soft law and disclosure to coverage mandates, and nanotech-era collaborative governance to “try-first” AI policy (2025-07-23).
• Care. From 2025 “effect” is “what we care for” (2025-05-18), and deploying institutions carry a “duty of care” (2025-11-09).

12. Tensions, ambiguities and gaps#

1. Two definitions, no integration rule. He uses both “probability of harm” (2019, 2023, 2024) and “threat to value”, and says the second extends the first. But he never shows how probability, exposure and dose–response apply to threats to dignity, identity or belief. His 2023 attempt to do so for AI ends by noting “the lack of even the beginnings of a framework”. The 2026 paper argues that value can be “named, mapped and watched — even where it cannot be measured”, which sets probability aside for those harms rather than integrating it. (Interpretation.)
2. Evidence standards for intangible harms. His 2019 insistence on weight of evidence and against “knee-jerk” reactions to single studies sits uneasily with his 2025–26 willingness to act on emerging, anecdotal or trajectory-based signals: lawsuits as “the very small tip of a very large metaphorical iceberg” (2025-11-09), and risk as what “might be possible given current trends” (2025-07-06). He does not say what evidentiary bar applies to cognitive or relational harms.
3. “No cause, no risk” versus tail planning. In 2023 he calls speculative scenarios hazards, not risks, without a causal pathway. Yet he supports edge-case planning “on the off chance” (2025-04-06) and small-probability research on cognitive risk (2026-01-10). The book’s Occam’s Razor discounts stacked assumptions, while its complexity chapters warn that small events trigger cascades. He does not reconcile the two epistemic moves (also noted in FFTF-F).
4. Whose value, and who does the valuing? The frame is “agnostic to particular worldviews”, yet deciding which value counts, and whose, is political. Its early form is enterprise-centred and recasts ethics as enlightened self-interest (“their success is intimately intertwined with how they impact … others”, 2024-12-17), so harms to people without leverage register only when they feed back to the firm. He acknowledges this only in the Fable-assisted 2026 paper (“not equally open to everyone”). The admitted subjectivity (2018-12-13) never becomes a procedure for resolving conflicts between values.
5. Risk professionals versus democratic definition. He holds that acceptability is set “by what people agree on” and that publics need no technical knowledge. He also appeals to “people and institutions who know a thing or two about risk” (2026-09-15) and warns about expert crowds regressing “to the mean” (2025-01-19). Who arbitrates between lay values, risk professionals and mainstream experts is left open.
6. Symmetry versus inevitability. Counting the risks of not innovating is a weighing principle. By 2026, “We can’t pause it” (2026-09-24) removes one side of the scale for AI. The frame’s formal symmetry sits alongside a practical assumption that some options are unavailable. The narrow companion-bot pause (2024-10-27) is the exception.
7. Anti-alarmism under a darkening tone. He keeps insisting that talking about risk is not fear-mongering. Meanwhile his descriptions of AI grow starker: “scariest”, “first technology of it’s kind” to slip into the mind. He does not say how his own heightened concern differs in method from the alarm he criticises. I infer the answer is specificity and mechanism; he does not state it.
8. Orphan risks: definitional drift. In 2018 they are “known knowns” treated as irrelevant. In 2020–21 they are hard to quantify and easy to ignore. In 2026 they are named but “unowned”, with no tools or accountability. The drift is modest, but it moves the concept from a cognitive failure (not noticing) to an institutional one (not owning). The 2026 apparatus is partly Fable-originated, which complicates any attempt to credit the 2026 version to him alone.
9. Analogy and its limits. The chemical and hazard–exposure grammar is his professional home. He also says frontier AI “defies analogy” (2026-01-22) and that past frameworks yield “categorical errors” (2026-09-24). His working resolution is that analogies serve as structure and mindset, not as templates. What remains of the chemical grammar for AI risk, in practice, is unstated.
10. Gaps. No worked method for prioritising among threats to different kinds of value, or for aggregating many small, dispersed harms (the “accumulative” pathway appears only in the 2026 apparatus). Little engagement with the risk-governance literature beyond IRGC, COMEST and Stilgoe et al.; Douglas and Wildavsky, Kasperson, Vaughan and ISO 31000 appear only in the Fable-assisted 2026 paper. No reported evaluation of whether the risk-innovation tools change outcomes. Value-based resilience (FFTF p.262) is never connected back to threat-to-value.

13. The most important sources for this thread#

1. 2016-01-11 thinking-innovatively-about-the-risks-of-tech-innovation. The founding statement of risk innovation and threat to value, the risk “market”, and risk landscapes.
2. 2016-03-02 how-risky-are-the-world-economic-forums-top-10-emerging-technologies-for-2016. Broadened value, future value, subtle versus tech-fixable risks, enthusiasm and fear as twin dangers.
3. Films from the Future (2018). pp.22–24 (risk innovation, threat to value), pp.118–122 (occupational risk and justice), p.174, pp.199–206 (plausibility), p.262 (value-based resilience), p.281 (Occam’s Razor), pp.289–290 (“Don’t Panic”).
4. 2018-12-13 tech-startups-orphan-risks (reposted 2023-11-15). The canonical definitions of orphan risks and threat to value, and reciprocity.
5. 2019-03-05 should-we-be-treating-algorithms-the-same-way-we-treat-hazardous-chemicals. Hazard versus risk, algorithmic exposure, weight of evidence, no zero risk.
6. 2019-11-01 how-to-build-a-better-brain-machine-interface-while-not-falling-at-the-first-hurdle. The landscape-mapping method, in place of an ethics critique.
7. 2020-07-30 life-on-mars-astrobiology-and-thinking-differently-about-risk. “more to risk than probabilities”, who bears harm, COMEST precaution.
8. 2020-11-05 risk-innovation-and-the-future. Risk as inevitable, outmoded risk ideas as a risk, the Nexus history.
9. 2023-05-31 existential-risks-of-ai. Catastrophe as mass loss of value, against extinction framing.
10. 2023-11-26 everything-youve-heard-about-ai-risk-is-wrong (with addendum). First principles, risk as a social construct, hazard–exposure tested on AI.
11. 2024-06-20 ilya-sutskevers-safe-superintelligence-rethink. No absolute safety, acceptable risk by agreement, who decides what “safe” means.
12. 2024-12-17 navigating-the-challenges-and-opportunities-of-advanced-biopreservation-technologies (with 2024-08-25 advanced-technology-transitions-model). Value versus values, existing versus future value, internal versus reciprocal threats.
13. 2026-05-10 do-not-do-this-with-ai. Safety message first, the limits of literacy, the “Luddite” charge.
14. 2026-09-15 will-ai-really-kill-us-all. The ten risks reaffirmed, risk talk versus fear-mongering.
15. 2026-07-16 orphan-risks-frontier-ai-maynard. Threat to value in frontier-AI governance. The apparatus is partly Fable-originated.