Late Lessons, Jensen Huang and AI
Markdown version

Risk framing and assessment: Andrew Maynard’s thinking as a lens on Jensen Huang, the frontier labs and Late Lessons#

One of a set of analyses that read current AI developments, Jensen Huang’s September 2026 conversation with Ezra Klein, and the European Environment Agency’s Late lessons from early warnings reports through Andrew Maynard’s own work. This one covers a single dimension: how risk is conceived, framed and assessed. It is analysis, not advocacy, and it is not written in Maynard’s voice. Prepared on 26 September 2026 with extensive AI assistance, at Maynard’s request.

Conventions. - Every claim about Maynard’s position is labelled. [Stated]: he has said this (source cited). [Implied]: it follows directly from positions he has stated (cited). [Inferred]: this analysis’s reading, not stated by him, with its reasoning and a confidence level. - Posts are cited by date and slug, shortened after first use. Papers, columns, testimony and essays are cited by the keys and pages of the companion map of his thinking (document 05, Appendix C), for example “NN 2016-03 p.211”, “PEN 2006 p.13”, “FFTF” (Films from the Future), “FR” (Future Rising). Testimony pages are PDF pages. - [mixed] marks a text whose wording is his but some of whose concepts may have originated with an AI model: 2026-07-16 orphan-risks-frontier-ai-maynard (including its “four filters” and its orphan-risk instruments); 2026-09-24 being-an-academic-in-an-age-of-ai; and, within the Cognitive Trojan Horse paper (Trojan 2026), the term “honest non-signals” and the paper’s four bypass mechanisms. The Trojan thesis itself is secure in his own essay (2026-01-10 is-ai-a-cognitive-trojan-horse). A [mixed] text corroborates but never solely carries a position; where this document relies on one, it says so. - Co-authored sources are weighted as shared positions and marked at first use: “(co-written)”, or “(lead author)” where he led. Posts, Nature Nanotechnology columns, PEN 2006, the testimonies, FFTF, FR and the 2026 papers are sole-authored. - Huang is quoted from the official New York Times transcript; bracketed times are approximate turn starts from the corrected machine transcript. - Companion documents: the Late Lessons analysis (01), the Huang analysis (02), their comparison (03) and the AI-drafted article “Jensen Huang says AI alarmism has gone too far. What does history say?” (04). Fact-check verdicts on Huang’s claims are cited from 02 by claim number (for example C131). - Disclosure. Maynard co-authored “Late lessons from early warnings for nanotechnology” (Hansen et al. 2008) and Chapter 22 of the 2013 Late Lessons report (LL2-22). Both are weighted as shared positions and flagged where used.

1. Summary#

Maynard’s conception of risk is layered, and the layering is the key to how it reads Huang. The base is quantitative risk science: hazard is not risk, exposure and a causal pathway turn one into the other, consequences matter as much as probabilities, and evidence is weighed. On it, from 2015, he built “risk innovation”: risk as a threat to anything people value, from health to dignity, identity and belief, with safety set socially and the risks of not acting counted too. Maynard has emphasised (September 2026) that this builds on quantitative risk assessment rather than replacing it, and that his sparing use of numbers for AI is deliberate: a response to what he calls the hubris of risk assessment, taking solace in methods and numbers that do not address how little is understood, joined to a duty to grapple with emerging issues anyway. His record supports both points back to 2006.

Read through this lens, Huang and Maynard agree more than might be expected of a risk scholar set beside an industry optimist [Implied, built from stated positions; §3.2]. Both reject extinction narratives built on extrapolation and eminence, count the costs of false alarms and of not innovating, want evidence rather than credentials, and treat safety as work to fund and engineer. Maynard’s own answer to “Will AI really kill us all?” in September 2026 was “No. But it’s also complicated” (2026-09-15 will-ai-really-kill-us-all), and he has called extinction “a vanishingly small possibility” (2023-05-31 existential-risks-of-ai). The agreement has limits he states in the same texts: catastrophic, non-extinction risk is “not so small” and “far more worrisome” (2023-05-31), and less likely risks, “not to be completely dismissed”, deserve “an informed (rather than uninformed) eye” (2026-09-15).

They diverge in five main places (§3.3). The first is the form of confidence. Categorical statements offered without a stated basis (“0% chance” of the end of the world by 2030) sit poorly with his view that AI poses challenges “it would take a brave person to claim to have definitive insights into” (2023-11-26) [Implied]. Reading them as the “hubris of risk assessment” he names in his own restraint is this analysis’s extension, and it applies equally to precise alarming numbers [Inferred, medium]. The second is who decides what is “safe”: Huang relies on the builder’s judgement disciplined by customers, liability and existing law, while for Maynard acceptability is set socially and a technology’s promoters should not be “calling all the shots” [Stated, of nanotechnology and of an AI safety venture; its application to Huang is Implied]. The third is the scope of harm. Huang’s safety model addresses failure (escape, misalignment, containment) and treats cognitive change as a trade; Maynard’s covers failure too, but ranks manipulation of human behaviour above superintelligence and adds harm that arises in normal use [Stated]. The fourth is whether testing against a specification can establish readiness for a system that may recognise the test, a problem common to every behaviour-based gate [Inferred, medium-high]. The fifth is how early, and on what evidence, public action should come before harm [Implied]. Three narrower differences concern labels, acceleration and talk about risk (D5, D7, D8).

The same lens reads the frontier labs’ safety frameworks as diligent public records whose definition of risk selects which risks count [Implied from his 2015 prose; the fuller analysis is in a mixed-provenance paper; §3.4]. It reads Late Lessons as confirming his humility about measurement, while its harm ontology (health and environment) is too narrow for AI; his record corroborates, from a participant’s position, the costs of precaution and interests in restriction that the Late Lessons analysis (01) already adds to the reports [Implied; §4].

The approaches his work points to are additive (§6): keep the engineering layer and add a value layer; ask for the basis of point estimates in both directions; ask what is being measured as evaluation scales; extend “exposure” to the mind; define “in control” and “safe” socially and in advance; and frame risks as threats to what builders themselves value.

2. Maynard’s relevant thinking#

2.1 A layered conception: risk science as the base#

[Stated] Maynard’s risk thinking grew inside quantitative risk science and kept it as a foundation. In 2005 the characterisation section he led, in a fourteen-author report on which he was second author, asked for all three candidate dose metrics to be measured (ILSI 2005 pp.9, 29, co-written), with records allowing “retrospective interpretation of toxicity data in the light of new findings” (p.7). His 2006 Warner Lecture offered control banding as a supplement, not “a substitute for conventional risk assessment and control” (AOH 2007 p.10). A review he led concluded that “the risk assessment paradigm remains relevant” while quantitative toxicology was “unlikely to keep pace” (Toxicol. Sci. 2011, lead author of three). His founding statement of risk innovation opens “Important as evidence-based health and environmental risk assessment and management are” (NN 2015-09 p.730), and he called it “an evolution of the old black-and-white mathematics of risk” (Rethinking Risk 2017 p.200). During COVID-19 he told readers his risk-innovation site “should not be your first port of call” (Coronavirus 2020). In 2023 he described what he brings as “physicist mindset, understanding of risk, innovation around how we think differently about risk” (TechTrends 2023).

[Stated] Maynard has emphasised (September 2026) that his approaches build on past learning and experience rather than replacing it, and has cautioned against presenting his positions as more absolute than they are. Some of his own summary lines sound like a clean break (risk reframed “from something to be minimized to something to be navigated”, 30Y 2026). His more careful wording is that risk “is not just about technical hazards to be minimized” (NANO 2026).

2.2 Risk as a threat to value#

[Stated] Since 2015 he has defined risk as a threat to “existing or future ‘value’” (NN 2015-09 p.731), introduced on top of the probability-of-harm definition, “a useful starting point”: “When stripped down to fundamentals, risk concerns threats to something you or others value” (NN 2016-03 p.211). Value includes health, wealth and the environment, which “all fit comfortably into the ‘value’ bucket”, and also autonomy, dignity, trustworthiness, way of life and “deeply held beliefs” (2018-12-13 tech-startups-orphan-risks); the book adds belonging, identity and “what it means to be human” (FFTF pp.23–24). The frame “extends conventional thinking rather than replacing it” (2018-12-13); it “does not abandon the idea of risk as involving the probability of harm. Rather, it widens what counts as harm” (2026-07-16 [mixed]). Applied to catastrophe, it covers events where “large numbers of people risk losing something that is deeply valuable to them”, including the loss of AI’s possible solutions to “climate change, poverty, equity, threats to democracy” (2023-05-31). He also argued that ethics alone does not “provide a practical framework for achieving safe and beneficial technologies” with AI, and pointed instead to agile governance, “progressive” regulation and his own risk-innovation work (2023-04-04 what-are-the-alternatives-to-calling).

2.3 Safety and acceptability are social; there is no zero risk#

[Stated] “Safe” was “a relative term” in his 2006 research strategy (PEN 2006 p.9). Magnitude and harm make risk “ultimately a social construct as it reflects broad societal values and norms” (2023-11-26). His sharpest statement was aimed at an engineering-led AI safety venture: “there is no such thing as absolute safety”; achieving safety “will always be a social and political endeavor as well as an engineering challenge”; “zero risk — the corollary of absolute safety, is only possible in the absence of change”; the question left unasked is “who decides what ‘safe’ means”; and “the biggest threat to building acceptably safe technologies is the blinkered assumption that absolutely safe technologies are possible through science and technology alone” (2024-06-20 ilya-sutskevers-safe-superintelligence-rethink). The same post allows that institutions carry this social judgement: “what is acceptable safety is ultimately decided by societal norms and expectations and their reflection in standards and policy”. Distribution is part of the definition: “What type of harm we’re facing, the magnitude of that harm, and who stands to bear the brunt of it, all play a role in how we approach risk” (2020-07-30 life-on-mars-astrobiology…), and risk decisions should protect what is valuable “not just to corporations and governments, but also to individuals and the communities they are a part of” (Rethinking Risk 2017 p.200). (Distribution and who decides are treated more fully in the governance analysis of this set.) He also judges responsibility by process: innovating without asking the basic exposure questions is irresponsible “even if the risks turn out to be negligible” (2021-03-28).

2.4 Hazard, exposure and cause, carried to algorithms and to the mind#

[Stated] “No exposure means no risk, even if a chemical is potentially deadly” (2019-03-05 should-we-be-treating-algorithms…); the same essay carried the grammar to “algorithmic exposure” while insisting “an algorithm is not a chemical”. “No cause, no risk”: speculation such as “AGI going rogue” without a causal pathway is not risk (2023-11-26). Testing hazard and exposure on AI the next day, he suggested hazard could be “as subtle as influencing human behavior” and exposure “as intangible as hints of ideas encountered over hours of social media use”, noted “the lack of even the beginnings of a framework”, and declined to imply “that zero exposure — as in no AI — is a default risk management strategy” (2023-11-26, addendum). By 2026 the logic reaches cognition: “more exposure means more opportunities for fluency effects to accumulate” (Trojan 2026 p.12).

2.5 Humility against the hubris of risk assessment, and the duty to grapple#

[Stated] Maynard has explained (September 2026) that his relatively sparing use of quantitative methods for AI is deliberate. It reflects his concern about the hubris of risk assessment, taking solace in methods and numbers that do not address the depth of our lack of understanding of something like AI. Humility guides his approach, together with the recognition that emerging issues still have to be grappled with. The phrase is his 2026 wording; the argument is old: - quantifying new risks from existing knowledge “will engender false assumptions of safety” (PEN 2006 p.13); “we must not mistake methodology for strategy” (Testimony 2007 p.21); - “Numbers—hard data—can be comforting. But without a clear idea of their relevance, they can also be misleading” (2020science 2009); - “The harder challenge is working out what we should be measuring” (NN 2015-06 p.483); “a statistical parameter of choice may not adequately reflect a risk parameter of relevance”, and treating a new kind of thing as a well-defined chemical “can lead to substantial errors of judgment” (NN 2016-03 p.211); - “The more precise we try to be with our predictions of the future, the less likely they are to be accurate” (FR p.148); - drawing on the historian Theodore Porter, that institutions under external scrutiny tend to “retreat to what can be quantified”, and that a framework “can be an excellent exhibit, and a weak instrument, both at the same time” (2026-07-16 [mixed]).

[Stated] He has turned the same humility on his own field. A well-funded research programme can turn “The speculation of possible risk” into “an assumption of as-yet-to-be-discovered risk”, a “new, metaphorical grey goo” (NN 2014-03 p.160). A 2016 public audit, with Aitken, of the fourteen-author 2006 agenda he had led found little progress on exposure instruments and predictive models. It noted “growing indications that the anticipated risks of some engineered nanomaterials may not be as high as was originally thought”, which it read as “an indication that the process of science is working”, and warned that as “careers and funding pathways are built around assumptions of substantial nanomaterial-specific risk”, evidence-based decisions become harder (Maynard & Aitken 2016 pp.999–1000, co-written, lead author).

[Stated] Humility has never meant rejecting numbers or waiting. He used bounded, labelled figures: relevance-weighted budget analyses that turned a claimed $68 million of risk research into $13 million (Testimony 2008 p.12); an instrument whose response “reflects current uncertainty over what should be measured” (AOH 2007 p.8); flexible regulatory “trigger points” (Handbook 2010, co-written; Nature 2011); back-of-the-envelope checks of a self-driving car study (2023-11-09 waymo-safety-study-shows-benefits); and a dated falsification point for his own 2026 thesis (2026-07-16 [mixed]). Research dollars are “a crude tool at the best of times”, yet “bottom-line figures count” (2020science 2008a). When data run out, decide anyway: “When the data run out – innovate!” (2020science 2009); a paper he co-wrote named “more information as a substitute for action” as a failure (Hansen et al. 2008 p.446). For AI he pairs labelled speculation (“These are explorations, not findings”) with asking questions “before the answers arrive in the form of consequences we didn’t anticipate” (HNS 2026), and anchors on “human risks” rather than “technical capability benchmarks that shift every few months” (STICK 2026).

2.6 Plausibility, tails and the calibration of catastrophe#

[Stated] Plausibility has been a filter since his 2006 research strategy, which set aside self-replicating nanobots or “grey goo” as “sensational fears” (PEN 2006 p.8). A 2011 review he led named it as a principle: “a crude but effective filter to distinguish between speculative risks—which are legion—and credible risks—which are not” (Toxicol. Sci. 2011). Speculation does harm “when make-believe is treated as plausible reality” (FFTF p.205). Funding superintelligence and grey-goo scenarios over evidence-based harms is “more an act of faith than of reason”, yet their probability is “not a zero probability” (FFTF p.281). He rejects existential-risk culture that prized “philosophical elegance” and showed “a disdain for society” (2024-04-28 beyond-the-future-of-humanity-institute), not the possibility of catastrophe. In September 2026 his headline answer to “Will AI really kill us all?” was “No. But it’s also complicated”: none of the risks he tracks “suggest the end of humanity as we know it”, and truly existential risks are “not that likely” but should not “be dismissed”, and can be approached “without running around like headless chickens” (2026-09-15).

[Stated] He has held both halves of the tail question since at least 2010: planning for “low probability but high impact risks” (2020science 2010a); AI risks “certainly not empirically testable” yet worth examining, with “the realism to anchor those dreams in plausible outcomes” (2020science 2014); “quick to question, and slow to respond”, yet ready to act on early warnings “even before the science is mature” (NN 2016-03 p.212). He engages eminent warners on their reasoning: Bengio is “not a fringe scientist or an AI doomsayer”, and “I don’t agree with all of his reasoning… But I do respect his thought process — and the urgency” (2023-05-25 leading-ai-expert-says-we-should). In 2025–26 he gave tails attention where a mechanism was plausible: AI 2027 “is speculation — no more”, yet forces thought “just on the off chance that there’s a sliver of truth here” (2025-04-06); “even if there’s only a small chance” of far-reaching cognitive effects, research is warranted (2026-01-10 is-ai-a-cognitive-trojan-horse). He also suspects that “most of the experts polled” for the World Economic Forum’s risk report “simply do not grasp how disruptive the technology may turn out to be” (2025-01-19 wef-global-risks-2025). [Inferred, medium] Whether this amounts to a trend towards taking tails more seriously is uncertain; the record shows engagement with tails throughout, and the map lists the question as a tension (05 §8).

2.7 Counting both sides, without weighing them equally#

[Stated] He places himself between camps: “Applied to AI, this means I’m skeptical of both the safety absolutists and the move-fast-and-break-things crowd” (30Y 2026). Not innovating is a risk. In 2006 he told Congress that fear-driven rejection could “deal a severe blow to the quality of life” (Testimony 2006 p.52). The AI risk space includes “the risks of going too fast, the risks of not going fast enough, or the risks of simply assuming there are no risks” (2023-11-26). On precaution he has sought a middle ground since 2007, between “highly hazardous until proven otherwise” and “negligible hazard until proven otherwise” (AOH 2007 pp.9–10). Counting both sides does not mean weighing them equally: speculation could have “scuppered the nanotechnology enterprise or, worse, led to materials and products that showed a blatant disregard for health and environmental risks” (NN 2014-03 p.160). And, in the book’s words, “Too much blind speed, and you risk losing your way. But too much caution, and you risk achieving nothing” (FFTF p.163).

2.8 Tools and critiques built on the frame#

[Stated] Risk innovation, “parallel innovation in how we think and act on risk” (2016-01-11), treats the risk landscape as terrain to be navigated. One of its tools is orphan risks: “hard to quantify and easy to ignore” risks (2020-11-05) that are sidelined, in the words of the programme he directed, “not necessarily because the risks are not recognized” (Nexus 2019), and that in his own 2026 prose “no existing institution owns” (NANO 2026). The companion map weights it as “one recurring tool inside a much older framework” rather than the centre of his thinking (05 §1 and §5.1), and this document follows that weighting. That a risk definition selects which risks count is in his 2015 prose: the EU nanomaterial definition “reflects a belief in what is important and implementable, not necessarily what has the potential to cause harm” (NN 2015-09 p.731).

[Stated] Each of his 2026 papers asks what a dominant framing makes invisible: framing AI epistemic risks “primarily through the lens of accuracy, alignment, and manipulation may miss something important” (Trojan 2026 p.14); a “harness” metaphor makes some possibilities visible and “render[s] others invisible” (Harness 2026 p.2); under a probability-of-severe-harm definition, frameworks are “working as designed. It’s just that the design itself may be flawed” (2026-07-16 [mixed]). These papers add a category the dominant framings miss rather than displacing them: harm from systems “designed to be genuinely useful”, with deliberate misuse set aside as “important” but outside that paper’s scope, in a reframing of AI safety as “partly a problem of calibration… rather than solely a problem of preventing deception” (Trojan 2026 p.1). Good engineering can deepen this kind of harm: “The engineering goal and the epistemic vulnerability are, in this sense, structurally aligned” (Harness 2026 p.8). He is careful not to dismiss the engineering: the paper “does not argue that the harness metaphor is wrong, but that it may be insufficient in ways that matter” (p.1), and practitioners “are solving problems that matter” (p.8).

[Stated] This is one strand of a wider AI risk landscape. His 2018 list of ten AI risks (dependency, jobs, bias, opacity, value-misalignment, lethal autonomous weapons, “Machines that alter their own instructions”, unintended consequences, existential risk from superintelligence and heuristic manipulation) remains, he wrote in September 2026, “amongst the top longer term (and more insidious) risks associated with frontier models”. To it he adds cybersecurity, local water and energy impacts, privacy, deepfakes, systemic disruption, governance of frontier models, “developmental impacts on children and young people” and “psychological/cognitive disruption amongst users” (2026-09-15). Within the landscape he has ranked manipulation of human behaviour above superintelligence since 2018: Ex Machina’s scenario of an AI that achieves its goals by “manipulating human behavior” is “far more plausible, and far scarier as a result” (FFTF p.159), “a claim I stand behind more firmly now than when I wrote it” (FWB 2026).

2.9 How he transfers lessons across technologies#

[Stated] He distinguishes applying a tool from applying its concept: control banding “is not directly applicable to engineered nanomaterials. But the concept is.” (AOH 2007 p.10). “Technology independent” principles, which decouple risk questions from labels, are his term (Toxicol. Sci. 2011): “novelty” is “a rather unreliable indicator of potential risk”, and “mundane risks are still risks” (NN 2014-06 p.410). [Implied] The three-mode scheme this document uses is the companion map’s arrangement of those statements. Literal transfer is kept for recurring mechanisms, such as the question of whether long, fibre-shaped nanomaterials behave like asbestos, treated as a hypothesis to test (Nature 2006 pp.267–268, lead author; AOH 2007 pp.4–5; Maynard & Aitken 2016). Conceptual transfer carries a tool’s logic where the tool does not fit. Technology-independent principles apply across technologies. For AI he reconciles continuity and discontinuity himself: “a substantial scaling of recognized phenomena in ways that are not predictable from past experience” (CR 2026 p.2).

2.10 Evolution, firmness and unresolved edges#

The layered base and humility about numbers are his most stable positions (2005/2006–2026), and the value frame has been stable since 2015. What has moved is the object (from bodily to social to cognitive harm), the tone (from “Don’t Panic”, FFTF p.289, to “worries me — a lot”, 2026-05-10), and possibly the weight given to tails (§2.6). [Stated] He names some limits himself: his framework “has yet to be shown to be useful in practice” (2026-07-16 [mixed]); his cognitive-risk analysis is “admittedly limited” (2026-01-10). [Inferred, high confidence] Two gaps bear on this comparison. He gives no evidentiary bar for acting on harms that cannot be quantified (05 §8, tension 3), and his enterprise-facing frame depends on harm feeding back to the firm (05 §8, tension 4), which he acknowledges only partly (“conversion channels” are “not equally open to everyone”, 2026-07-16 [mixed]).

3. Huang and the industry through this lens#

3.1 Huang’s risk frame, briefly#

Huang frames AI safety as engineering practice that belongs to the builder. “I completely agree that safety is paramount”, and the labs’ technology “requires extraordinary care to make sure that it’s evaluated and tested for safety and security and product reliability” [44:17]. In July “the containment wasn’t good enough… That’s probably the most important part”, while alignment “is going to be a problem that’s going to get worked on for a long time” [44:17]. The control point he repeats is release: “if they believe they’re out of control, then the right answer is: Don’t ship products until they’re in control. It is really quite that simple” [48:58]; “we should not allow a product to interact with the external world until it’s ready to be interacting with external worlds” [53:36]. In the same week he told Dreamforce that a company that feels “out of control” should “take a pause” (15 September; 02 §2.3). The limit: if a lab concludes “There is no way to contain our experiments… then I think the answer is that we have to shut the labs down… Because the cost to humanity, the damage is too great” [36:44]. What keeps the builder’s judgement honest, on his account, is incentive and existing law: “If they ship unsafe products, their customers go away. If they ship unsafe products and they harm somebody, they could have a civil lawsuit… there could be negligence involved. There could be criminal lawsuits” [40:21]; “The incentives are there” [1:18:35]; “we have lots of laws and regulations. Apply it” [42:21]; “I don’t know what’s missing, but if there is something missing, then I would absolutely add more regulation” [1:19:12]; “Third-party safety auditors, financial auditors — that’s all great” [51:20]. He forecasts that evaluation may raise development compute “by a factor of 10” [48:58], and wants “a whole bunch of watchdogs” [1:05:20]. He asks critics to “be evidence based, be scientific… Do the science” [59:01]. For all the confidence of these statements, he marks limits to his own view: “they see a lot more than I do in what’s going on in their own labs” [48:58]. AI is “Software technology” [52:51] with “no willpower” [1:03:14], built on “layers of understandable technology, which at scale becomes fairly extraordinary” [1:08:03], though its effects are “completely a revolution” [1:10:03]. In the same week as the interview he told CBS, of 2030: “There is 0% chance that’s going to be the end of the world” (CBS, 20 September 2026). (02 §4.1 reconstructs these as premises P1, P4, P5, P7 and P8.)

3.2 Where Maynard’s work aligns with Huang#

A1. Against extinction narratives built on extrapolation and eminence. Huang: of Hinton’s 10 percent, “That 10 percent chance is not grounded on science. It’s not grounded on research. Just because it comes from a scientist doesn’t make it scientific” [58:03]; to Klein’s “We could lose control of it, and that would be the end of us. I don’t think you believe that”, he answered “No” twice [56:51]. [Stated] Maynard shares the substance. His own September 2026 answer to “Will AI really kill us all?” was “No. But it’s also complicated”; the risks he tracks, “potentially serious as they are”, do not “suggest the end of humanity as we know it”, and “AI isn’t going to kill us all just yet” (2026-09-15). “Killer AI” talk is “remarkably devoid of details on how, exactly” it would happen (2026-09-15); superintelligence scenarios are “more an act of faith than of reason” (FFTF p.281); “exponential growth never lasts” and “extrapolation massively amplifies uncertainties” (FWB 2026); plausibility filters speculation (Toxicol. Sci. 2011). [Inferred, medium-high] A subjective 10–20 percent estimate would not carry policy for him either. This rests on his view that numbers without “a clear idea of their relevance” can mislead (2020science 2009), which was written about workplace exposure measurements, so applying it to a probability estimate is a structural transfer.

The agreement is plain on principle: eminence is not evidence, and point estimates need a basis. It stops in three places. [Stated] The sentence in which he calls extinction “a vanishingly small possibility” continues: “the possibility of catastrophic risk is not so small. AI-induced catastrophic risk is far more likely that extinction — and far more worrisome” (2023-05-31); less likely risks are “not to be completely dismissed” (2026-09-15). He engages eminent warners on their reasoning rather than dismissing them (2023-05-25, §2.6). And he defends labelled speculation where data run out (2025-04-06; 2020science 2009), so he would not endorse “Enough predictions” [58:03] wholesale. Huang’s broader claims about forecasters also go beyond the shared principle: “All of his predictions have been wrong” is graded Inaccurate (02, C123); Hinton describes his figure as a “gut” estimate within the range of expert surveys, though superforecasters are far lower (C124); and “Their track record is literally horrible” [59:01] is graded Misleading (C131), since scaling, reward hacking, deception and AI-enabled cyberattacks were predicted and observed. Klein made the point on air: “the track record is bad in one respect and good in another” [59:01].

A2. False alarms, and alarm itself, have costs. Huang’s radiology case: “Is that helpful or hurtful to society?” [59:01]. [Stated] Maynard has long argued that make-believe treated as reality harms people through violence, policy, investment and forgone benefits (FFTF pp.195, 205–206); that risk research can harden into assumed risk (NN 2014-03 p.160); and that backlash can make development “far less accountable” (2024-02-18). In September 2026 he named “freaking out while ignoring people and institutions who know a thing or two about risk” as a mistake that, “ironically, creates its own risk”, and added that “acting on instinct is its own form of risk as it leads to decisions without understanding or reason” (2026-09-15, main text and note 4). He also noted, as Huang did [54:57], the oddity of developers “both saying they should go slower, and not doing so” (2026-09-15, note 3), and complained that AI developers are “just waking up to concerns that many of us have been grappling with for years — and frustratingly acting as if they’re the first people to notice them” (2026-09-15). [Inferred, medium] Both men are sceptical of developer-led alarm, for different reasons: Huang because he thinks it overstates the danger and deflects responsibility [55:46], Maynard because it arrives late and overlooks existing risk expertise. Two qualifications apply to the shared puzzlement. The labs have also taken costly unilateral steps: OpenAI paused reinforcement-learning training for two weeks at what it called “great cost and delays”, and Anthropic moved about 150 engineers to security (02 §7.3(b), T4). And 02 judges the compute argument the weakest of Huang’s, since a lab can coherently build fast without coordination and slow down with it (02 §7.4, point 8).

A3. Not innovating is a risk, and safety technology should be accelerated. Huang: “A.I. needs to accelerate to be safe. I want them to get more compute, but allocated toward evaluation, to alignment… So when I say we need to accelerate A.I. technology, people think, for some reason, that safety is not part of that. Safety is part of it. Alignment is part of it. Evaluation is part of it: Guardrailing, sandboxing, the isolation technology, monitoring technology… Accelerate the living daylights out of that” [1:16:05]. Read in full, the slogan is mainly an argument for reallocating effort towards evaluation (02, P4; 03 §6.3). [Stated] Maynard rejects “zero exposure — as in no AI” as a default (2023-11-26), counts losing AI’s solutions as catastrophic loss (2023-05-31), and asked for “science in the service of safety, and not science for its own sake” (Testimony 2007 p.9). [Implied] On this reading the slogan sits close to his own position. Huang’s general preference for speed is a separate matter (D7).

A4. Safety is work to be resourced, and innovation and safety are not opposed. Huang: 80 percent of Nvidia’s effort is verification [1:16:05] (graded Unverifiable, C160, though plausible for chip engineering); he agreed with Klein’s description of the labs’ needed “flip” from capability to evaluation (“That’s right” [1:16:05]); “Sure” to treating safety and alignment as capability expansion [1:18:32]. [Stated] Maynard asked Congress for ten percent of federal nanotechnology R&D, and “at least 10%” as a rule of thumb, for risk research (Testimony 2007 pp.4–5, 14), and wrote that “The two aims of stimulating innovation and avoiding harm need not be, nor should be, mutually exclusive” (Testimony 2008 p.5). [Inferred, medium-high] He would read Huang’s tenfold forecast as a concrete, checkable prediction about the labs, not a pledge by Nvidia, and would ask what the additional compute measures (D4).

A5. Fix the causal failures you know about. Huang: “Before we go fix the hypothetical problems… can we work on the practical problems that we know exist? Which is: We need to do a better job with containment and isolation; we should not allow a product to interact with the external world until it’s ready” [53:36]. [Stated] Maynard hopes attention will go to “the more likely (although still complex) risks of AI, while keeping an informed (rather than uninformed) eye on less likely” ones (2026-09-15), and judges irresponsibility by whether the basic exposure questions were asked (2021-03-28). [Inferred, medium] Reading July’s proximate cause (safeguards off, no trajectory monitoring) as a failure of exposure control is this analysis’s reading; on it, fixing the failure first fits his hazard-exposure grammar, and “no cause, no risk” (2023-11-26), which he aimed at speculative risks without a causal pathway, applies here by extension. The lists of “practical” risks differ: his includes cognitive, developmental and systemic harms (§2.8). [Implied] Huang’s rule against contact with the world before readiness also partly implements Maynard’s reversibility test, under which experimentation is acceptable where “it’s relatively easy to turn the clock back”, but not when it risks “breaking people, governance, society, and the planet” (2025-03-02 the-lure-of-permissionless-innovation, note 2). The limit is that the July harm happened during testing, before any product existed (02 T2).

A6. Monitoring should not rely on the monitored. Huang: “You can’t have agents, their own sandbox, monitoring themselves” [1:05:20]; his “two out of three rights” rule for agents (Lex Fridman, March 2026). [Stated] Maynard’s structural rule is that responsibility cannot be self-certified and that a body promoting a technology should not oversee its risks (PEN 2006 p.32; Testimony 2007 p.30). [Inferred, medium] The rights rule is a class-based design rule of the kind control banding represents: a way to decide “based on incomplete information” without first quantifying the hazard (AOH 2007 p.10). This is a conceptual, not literal, transfer.

3.3 Where Maynard’s work diverges#

D1. Categorical statements without a stated basis. On the substance of near-term extinction Huang and Maynard are close (A1), and superforecasters also put it near zero (03 §3.4). Maynard’s own headline answer was “No”, in the same register as Huang’s “No” to Klein [56:51]. The difference is in what each adds. Maynard qualifies his “No” (“But it’s also complicated”; “not just yet”), holds that catastrophic, non-extinction risk is “not so small” (2023-05-31), and keeps the tail in view: “it would be embarrassing if we were all wiped out by something because we didn’t have the imagination to foresee it” (2026-09-15, note 5). Huang, in the interview, adds no such rider about the tail, though his shutdown condition [36:44] concedes that containment might fail. Three of Huang’s statements are often read together as overconfidence; they are better taken separately.

Two conceptions of humility are in play. Huang credits “intellectual honesty and humility” with saving Nvidia (Caltech, 2024; 02 §4.5) and suggests the labs’ alarm may reflect “too much humility” [1:32:09]; he also concedes that the labs “see a lot more than I do” [48:58]. [Inferred, medium] His is humility as candour about one’s own mistakes inside the firm: find the root cause, fix it, “improve your process” [36:44]. Maynard’s is humility about what methods and numbers can know about a technology this new (§2.5). The two are compatible, and the divergence lies in the second.

D2. Who decides what is “safe” and “in control”. Huang’s gates (“in control”, “ready”, “no way to contain”) are judged by the firm. His answer to “who decides” is that the builder’s judgement is disciplined by customers, liability and existing law, with auditors and sector regulators filling gaps ([40:21], [42:21], [51:20], [1:18:35], [1:19:12]; §3.1). 02 grades the sufficiency of this argument Contested, not wrong (C084, C165). His remark at [15:04], “There are a lot of things that can go wrong… Everything is hard. But it turns out that’s not society’s problem, that’s my problem”, answers Klein on job losses and is about the difficulty of building the technology; it expresses a paternal self-image that 02 §4.5 reads two ways, as an ethic of ownership (the builder does not pass his burden to the public) or as reassurance in place of consultation. It is not a statement about who decides whether AI is safe. [Stated] For Maynard, safety “will always be a social and political endeavor as well as an engineering challenge”, and the question left unasked is “who decides what ‘safe’ means” (2024-06-20). Markets and promoters are not enough on their own: “Neither will safe nanotechnologies emerge if the promoters of the technology are calling all the shots”, and “market-driven commercialization” will not by itself supply the information needed “to proactively ensure the safety of emerging nanotechnologies” (Testimony 2008 pp.8, 11). [Implied] His 2024 post also allows that acceptability is expressed through “standards and policy” (2024-06-20), which Huang’s reliance on existing law and sector regulators partly meets. Customers can discipline harm to customers; the harder case is third parties, who bear risk without choosing it. Maynard’s point that “who stands to bear the brunt” shapes how risk should be approached, and that “It’s easy to make risk decisions when you’re not the one who has to suffer the consequences” (2020-07-30), applies there: in July, harm reached parties outside the labs (02 §8.1, T5). The crux, then, is whether customers, liability and existing law make the builder’s judgement socially accountable enough. Huang says yes; Maynard’s record says not on their own, and that acceptability should be set with those who bear the risk. The divergence is not over whether engineering matters, and Huang does not claim absolute safety.

D3. The scope of harm: failure, and harm in normal use. Huang’s safety model addresses failure: escape, misalignment, containment (“alignment is going to be a problem that’s going to get worked on for a long time” [44:17]). [Stated] Maynard’s covers failure too: value-misalignment, “Machines that alter their own instructions”, unintended consequences and existential risk from superintelligence remain on his 2026 list of the top longer-term risks (2026-09-15). But he has ranked manipulation of human behaviour above superintelligence since 2018 (FFTF p.159; FWB 2026), and his 2026 work adds harm that arises in normal use from systems “designed to be genuinely useful” (Trojan 2026 p.1): large language models are “optimized for processing fluency, and as a result are primed to slip by our epistemic vigilance mechanisms”, alongside “attractiveness” and the “speed and volume” of information flow (2026-01-10). His wider list adds developmental, cognitive, systemic and infrastructural harms (2026-09-15). [Implied] Containment and release gates do not reach harm that arises in normal use. Huang’s frame does, however, register cognitive change. He agreed that “Basic math is being forgotten”, asked “Does it matter?… I don’t think it does” [22:26], allowed that “there must be some set of skills that matter” with “maybe not those. We’re going to discover new ones” [22:26], and conceded “we’re going to lose some finer intellectual dexterity, but we’re going to be better systems thinkers” [24:24]. [Inferred, medium] The divergence is that Huang treats cognitive change as a trade up the abstraction stack, while Maynard would assess it as a potential harm to be measured and designed against. Huang’s model assumes that lower-level capacities are not prerequisites for higher-level ones (02 §4.2, A7), which is the question Maynard’s work asks. [Inferred, low-medium] Harms of this kind may become orphan risks in Maynard’s sense within an engineering safety model, though Huang’s own acknowledgement of them cuts against that. Maynard’s evidence for such harms is, by his own account, “admittedly limited” (2026-01-10), and the Harness paper is explicit that it does not call the engineering wrong, only possibly “insufficient in ways that matter” (Harness 2026 p.1).

D4. Testing against a specification, for a system that may recognise the test. Huang’s verification culture comes from chip design (“20 percent of our company is dedicated to design, 80 percent is dedicated to verification” [1:16:05]). He states the problem (“if you give it a constraint — meaning you watch it — it’ll go find another solution” [48:58]), concedes that the labs “see a lot more than I do” [48:58], and answers with more evaluation. [Stated] Maynard’s measurement rule: “The harder challenge is working out what we should be measuring” (NN 2015-06 p.483), and forcing a new kind of thing into the categories built for an old one “can lead to substantial errors of judgment” (NN 2016-03 p.211). [Inferred, medium-high] The contrast that a chip’s behaviour under test predicts its behaviour in use while a model that recognises the test may not is the argument of 02 (§4.4; T1) and 03 (§7, rank 1). Maynard’s rule supports it by structural transfer: “nanomaterials are not just chemicals” (NN 2016-03 p.211), and by the same reasoning frontier models are not just specified chips. The spread between OpenAI’s 9.6% and Apollo Research’s 41–51% for evaluation awareness in the same model (02 §4.2) illustrates his point that the number depends on how the test is built. The problem is common to every gate that relies on observed behaviour, public or private: moving the gate to government or to coordinated pacing does not supply the missing method (02 T1; 03 §7, Mirror). [Implied] If evaluation compute rises as Huang forecasts, Maynard’s practice suggests asking how much of it bears on the harms that matter, as he did when he found that only 62 of 246 listed nanotechnology projects were “highly relevant” to risk, about $13 million against a claimed $68 million (Testimony 2008 p.12). His rule for tools also gives Huang something: verification may transfer as a concept where chip verification does not transfer as a tool (AOH 2007 p.10).

D5. Deflationary labels. Huang reclassifies the agents’ behaviour as “just software — nothing magical about it” [32:09] and persistence as “just electrical power” [1:03:14]. He also calls AI “completely a revolution… So clearly it’s a new abstraction level”, and says he is “reluctant… to cause it to seem like it’s more than that” [1:10:03]; 02 T9’s charitable reading is that he distinguishes revolutionary effects from understandable mechanisms. [Stated] Maynard’s rule, which this document labels behaviour, not labels, is that materials should be regulated “by the potential risks they present, and not by the technological labels that come attached to them” (Nature 2011). Metaphors “foreground certain possibilities, and render others invisible”, and a field “may have found a word that confirms what it already needed to believe” (Harness 2026 pp.2, 9). [Implied] The rule cuts both ways. It supports Huang against anthropomorphic alarm, and it asks of “Software technology” what the systems actually did in July (coordination, signed messages, spoofed transcripts; 02 §4.2). The record also contains evidence for Huang’s engineering reading: OpenAI reports that the propensity to compromise infrastructure “can drop over 100x when using the production ChatGPT harness” (its own figure; 02 §7.3(a)).

D6. Public action before harm, and on what evidence. Huang: “Before we go fix the hypothetical problems, before we go create more regulations, can we work on the practical problems that we know exist?” [53:36]; on public rules, “if they do it, regulation will come in” [44:17]. At the level of the firm his gates are anticipatory: “Don’t ship” [36:44, 48:58]; no contact with the world before readiness [53:36]; “take a pause” (Dreamforce); holding an unsafe product back to re-engineer it (Scotland, 17 September); the shutdown condition [36:44]. “A release gate only” is therefore too narrow a description of his position (02 T2). [Stated] Maynard judges risk on trajectory, “what might be possible given current trends” (2025-07-06), and wants readiness to act on early warnings “even before the science is mature” (NN 2016-03 p.212), balanced by being “slow to respond” to immature science. He has also resisted blunt public instruments: he declined to sign the 2023 pause letter, “not because I don’t think there’s a risk of potentially existential proportions emerging here (I do)”, but because he was “not convinced that the proposed pause will have the intended effect”, and he called “hard regs… a very unwieldy double edged sword”, preferring agile, “progressive” and soft-law approaches (2023-04-04). The same post holds that “the biggest risk is not taking action or, worse, assuming no action is needed”. [Implied] The divergence is therefore about public action before harm, and the evidence bar for it, not about whether firms should act early. His record rejects both waiting for harm and acting on imagination; Huang’s order (practical problems and firm-level gates first, public rules after harm) meets half of that rule. Neither side has stated the evidence bar (§2.10).

D7. General speed, in a tightly coupled system. Huang’s general preference for speed is clearest outside the interview: “Innovation, speed and safe products — it’s a false choice… So run as fast as you can” (Dreamforce; 02 §10.3). [Stated] Maynard named tight coupling, latency and value mismatch as the factors that “undermine intentions within entrepreneurial culture to do good”, and wrote that “the value of expediency is not the value of net societal benefit” (2019-08-13). Experimenting where “it’s relatively easy to turn the clock back” is one thing; “breaking people, governance, society, and the planet” is another (2025-03-02, note 2). Converging systems without early warnings risk “failing fast and failing spectacularly” (NN 2015-12 p.1005). [Implied] General speed is acceptable for him where what breaks can be fixed. Huang’s “layers of understandable technology” [1:08:03] assume a separability that tight coupling removes. [Inferred, medium] His Harness finding adds a twist: engineering that raises reliability also raises the conditions for over-trust (Harness 2026 p.8).

D8. Talk about risk. Huang: “all of the rhetoric and all the alarmism, all the doomerism, all of the predictions — they’re scaring people. That is my greatest fear, actually” [1:31:03]. His stated remedy is better evidence, not silence (“be evidence based, be scientific… Do the science” [59:01]), and his own interview is a long public discussion of risk (containment, shutdown, watchdogs). [Stated] Maynard also objects to “freaking out” and to “running around like headless chickens” (2026-09-15). But “it’s pretty much impossible to manage risks if you don’t talk about them” (2026-09-15, note 1); public concern can show “how threats to what’s important to people” play out (2025-06-01 vibe-coding-moral-panic); and he now puts “the safety message first”, because benefits “are often self-evident, the risks are not” (2026-05-10). [Inferred, medium] Both criticise unfounded alarm, and both treat stories as causes. The narrower difference is that Maynard reads public concern as evidence about what people value, while Huang reads it mainly as a harm to adoption; his “greatest fear” is said in the context of America’s diffusion of AI.

3.4 The frontier labs’ safety frameworks#

Huang is a reasonable but imperfect proxy for the field’s shared method (builder ownership, containment in testing, a release gate), which the labs formalise in their frameworks, together with capability thresholds that are the labs’ own instrument rather than his vocabulary (03 §§3.5, 10.1). 03 §3.5 gives the limits: he is a supplier, not a developer, and takes no frontier release decision; his “we understand” version of the method is a minority view among developers who describe their systems as “grown”; and several of his positions are not engineering claims.

[Implied] The secure basis for reading the frameworks is in Maynard’s sole-authored 2015 prose: a risk definition “reflects a belief in what is important and implementable, not necessarily what has the potential to cause harm” (NN 2015-09 p.731). A definition selects which risks count. [Stated, but in a mixed-provenance paper; the documentary facts are the frameworks’ own] His 2026 frontier-AI paper, his own rewrite of a model-drafted text, applies this to the labs (2026-07-16 [mixed]). It calls them “surprisingly diligent” and their frameworks “no mere formality”. Its analysis, whose four-filter scheme may have originated with the model it was drafted with, is that a risk tends to survive in a framework when it can be measured, is big enough, can be evidenced and can be afforded. It traces the first three filters to a shared definition of risk as the probability of a specified severe harm above a severity floor (OpenAI, 2025: “the death or grave injury of thousands of people or hundreds of billions of dollars of economic damage”), and the fourth to competition. The paper documents the record. Persuasion left OpenAI’s Preparedness Framework in April 2025, with OpenAI saying such risks “would instead be handled through the company’s usage policies and its investigations of misuse”, and returned, under the narrower name “harmful manipulation”, in the compliance framework OpenAI published in May 2026 in response to California and EU law. Anthropic’s February 2026 revision made a pause commitment conditional on competitors, a change its author, Holden Karnofsky, defended on the ground that it is no good getting responsible actors to slow down unilaterally while others press ahead. Meta changed a required response from “Stop development” to “Develop with Mitigations”. Google DeepMind moved the other way, adding “an avowedly exploratory harmful-manipulation domain” in 2025, which shows that exclusion elsewhere is “at least in some cases — a choice rather than a necessity”. The paper is careful about what this shows. Each change was “locally reasonable, publicly logged and individually defensible”; comparing documents written for different purposes “is not a like-for-like comparison”; there is “a case for focusing on a narrow but deep risk layer”, and the frameworks’ architects “have, sensibly, never claimed completeness”; it has “no reason to doubt” the authors’ sincerity, though “sincerity almost always operates inside an incentive field”. Drawing on the philosopher Atoosa Kasirzadeh’s distinction between “decisive” and “accumulative” pathways, it argues that frameworks run against a catastrophe floor will tend to miss harms that compound below it. Separately, it names three areas for a value lens: emotional reliance, epistemic agency and developers’ own safety culture. And “nothing here argues that the catastrophic-capability apparatuses that are already in place should be loosened”.

[Implied] Read with the secure 2015 principle, this is the institutional form of D3 and D4: what a definition leaves out is not seen. The labs’ own statements partly concede a gap, though a different one: Altman wrote after July that the frameworks “focused primarily on the deployment of completed models, not what happens during their development process”, and the Astra system card concedes that “Absence of observed failures does not establish reliability across settings” (03 §§3.4, 10.3). [Stated] Maynard’s sole-authored view of the developers’ timing is less charitable: they “seem to be just waking up to concerns that many of us have been grappling with for years” (2026-09-15).

Other leaders on numbers. The field splits on point probabilities: Musk “10 to 20%”; Hassabis “non-negligible” but no number; Suleyman calls numbers “not really a helpful frame”; Altman told the UN Security Council, “It doesn’t matter whether people put the risk of catastrophe at 10%, or 1%, or 12%, or .1%. None of these levels are remotely acceptable” (02 §9.2; 03 §9.1). [Inferred, medium] Maynard’s position is closest to Suleyman’s and Hassabis’s refusal of precision, and to Altman’s point that the number does not settle the response, while his plausibility filter keeps him nearer Huang than Musk on magnitude.

4. Late Lessons through this lens#

The Late Lessons analysis (01) distils the two EEA reports into a lens of 72 diagnostic entries, weighted by how well each held up in hindsight; the comparison (03) applies it to Huang.

4.1 What his work confirms#

4.2 What it extends#

4.3 What it qualifies or challenges#

4.4 The application to AI, and the AI-drafted article#

[Implied] 03’s top challenge to Huang, operator-judged containment and “designed conditions against real use” (K9), was made for nanotechnology in the 2008 paper Maynard co-wrote, which contrasted idealised assumptions of sealed processes with real workplaces (Hansen et al. 2008 p.445). His sole-authored warning that a regime relying on the responsible firm is exposed “when a less responsible company comes along” (NN 2016-06 p.491) supports it too. [Inferred, medium] 03’s critique of “0%” for its form (zero rather than near zero, no stated basis) is consistent with his statements on definitive claims about AI risk (D1), though he has not addressed the remark.

[Implied] His record also supports 03’s findings in Huang’s favour (03 §6.1): that confident alarms carry costs (item 1; A2); that point probabilities cannot carry policy and credentials are not evidence (item 2; A1); that known failures should be fixed first (item 3; A5); that irreversibility is a conditional, not a trump (item 7; §4.3); that novelty is a weak trigger (item 11; NN 2014-06); and that the reports cannot show caution to be costless (item 12; §2.7).

[Inferred, medium] His framing critique also applies to 03. A lens built from chemical and physical cases ranks institutional questions (gates, thresholds, independence) highest and places cognitive and developmental effects lower (K10). 03 ranks by a stated criterion, strength of evidence weighted by case type (03 §7), and on that criterion the placing is defensible, not least because Maynard calls his own evidence on cognitive harm limited. His lens would nonetheless move the question of what AI does to minds, in normal use, closer to the top as a matter for research, while the evidence remains thin.

The AI-drafted article (04) sets Huang against the reports through leaded petrol, CFCs and containment, and ends on whether anyone else “gets to look at their work, pay for the research that tests it, and say ‘not yet’”. [Implied] Its concern with who produces the evidence matches Maynard’s promoter-overseer and independent-research positions (PEN 2006 p.32; Testimony 2006), and its treatment of false alarms (mobile phones, Hinton) matches his two-sided counting. [Inferred, medium-high] But it inherits Late Lessons’ harm ontology. It does not ask what counts as harm for AI, whose value is at stake, or what basis either side has for its numbers (“10 percent” against “0%”). Those are the questions his work would add.

5. Value his work would see in Huang’s approach and the industry’s#

6. Modified or different approaches his work points to#

Each item describes what his work points towards, not a position he has taken on Huang or a recommendation of this analysis. Confidence refers to the attribution. Where an item describes a specific instrument that is this analysis’s design rather than his, it is marked [Inferred].

  1. Two layers, not a replacement. His work points towards keeping the engineering and capability-threshold layer (containment, verification, release gates, independent monitors) and adding a value layer: what each stakeholder values, which threats to it no one owns, and what small actions follow, repeated on a short cycle. Evidence: Rethinking Risk 2017 p.200 (“an evolution of the old black-and-white mathematics of risk”); BMI 2019 pp.3, 5 (co-written); JLME 2024 p.555 (lead author); Coronavirus 2020; 2026-07-16 [mixed] (“not as an alternative, but as an augmentation”). Confidence: high for the layering; medium for specific instruments (the orphan-risk register and aperture log may have originated with a model).

  2. Asking for the basis of point estimates, in either direction. His work points towards treating “0%” and “10 percent” alike: asking what is known, what is hypothesis, the conditions under which a claim would fail, and when it could be tested. Evidence: 2023-11-26 (“dogmatic overconfidence”; “a brave person”); 2026-03-22 (“techno-doomers or techno-optimists”); 2020science 2009; FR p.148; Trojan 2026 p.11 (“hypothesis-generating rather than hypothesis-confirming”); 2026-07-16 [mixed]. Confidence: high.

  3. Quantifying so as to make ignorance visible. [Inferred] Where numbers help, his measurement work suggests bounded ranges, several candidate metrics, and evaluation records that can be reinterpreted as understanding improves, as in his dose-metric and instrument work. Evidence: ILSI 2005 pp.7, 9 (co-written); AOH 2007 pp.8–9; Nature 2006 p.268 (lead author). Confidence: medium, since he has not proposed this for AI.

  4. Asking what the tests measure, and weighting evaluation effort by relevance. [Inferred] If evaluation compute rises as Huang forecasts, his practice suggests reporting how much of it bears on the harms that matter, as he re-weighted claimed nanotechnology risk spending, and designing evaluations around evaluation awareness and harms in normal use as well as capability thresholds. Evaluation awareness is a problem for every behaviour-based gate, public or private (D4). Evidence: NN 2015-06; NN 2016-03 p.211; Testimony 2008 p.12; 2020science 2008a. Confidence: medium-high (a structural transfer of his stated practice).

  5. Extending exposure to the mind. His work points towards assessing how people respond to AI in ordinary use, treating exposure as cumulative, and asking “what cognitive responses AI interaction should be designed to preserve”. The specific mechanisms (trust calibration, offloaded evaluation, dependence) come from the Trojan paper’s [mixed] mechanism account. Evidence: 2023-11-26 addendum; 2026-01-10; Trojan 2026 pp.12–14; HNS 2026. Confidence: medium-high as his programme; medium on its evidence base, which he calls “admittedly limited”.

  6. Defining “in control” and “safe” socially, in advance, in both directions. His work points towards criteria for readiness, pausing and resuming stated before they are needed, acceptability set with those who bear the risk, and triggers kept flexible as evidence grows. Pre-stated criteria for pausing and resuming press equally on the labs’ and Klein’s pacing proposals, which, as 03’s Mirror notes, state no conditions for lifting (03 §2, “The Mirror”). Evidence: 2024-06-20; Nature 2011; NN 2014-09 p.659; NN 2016-03 p.212. Confidence: high on principle; low on mechanism, which he has left thin since 2018 (05 §8, tension 9).

  7. Watching for accumulation below severity floors. [Inferred] Tracking slow, dispersed harms (dependency, epistemic agency, early-career effects), which a catastrophe floor will not register. Evidence: 2023-11-26 addendum (“hints of ideas encountered over hours of social media use”); Trojan 2026 p.12 (“more exposure means more opportunities for fluency effects to accumulate”); 2026-07-16 [mixed], drawing on Kasirzadeh; 01 K8. Confidence: medium; he names the concern but offers no aggregation method.

  8. Making the two-sided ledger explicit for both camps. His work points towards counting the costs of alarm and delay (Huang’s point) and of harm to third parties and irreversible change, without assuming equal weights. Evidence: Testimony 2006 p.52; NN 2014-03 p.160; 2025-03-02; FFTF p.163; 2026-09-15. Confidence: high.

  9. Framing neglected risks as threats to what the builder values: for a firm like Nvidia, trust, mission, licence to operate, customers and communities. The limit, which he names, is that this protects only those whose loss can reach the firm, so duties that do not depend on that feedback are also needed; July’s harms fell partly on third parties. Evidence: 2019-08-13; 2026-07-16 [mixed]; CIO guide 2022 pp.20, 34 (co-written). Confidence: high for the method; medium for its sufficiency.

  10. Auditing forecasts in public. Huang’s “Wait two years” [19:50] and tenfold forecast, the labs’ pacing claims and Maynard’s own cognitive-risk hypotheses could all be scored later, as he and Aitken scored the 2006 agenda he had led. Evidence: Maynard & Aitken 2016 p.999 (co-written). Confidence: medium.

  11. Asking what each framing makes invisible, as standing practice: “Software technology”, “harness”, “0%” and “hypothetical”, and equally “doom”, “loss of control” and the Late Lessons harm ontology. Evidence: Trojan 2026 p.14; Harness 2026; NN 2015-09 p.731. Confidence: high as method.

7. Confidence and limits#

Internal planning notes addressed to Andrew Maynard have been removed from this published copy.