Late Lessons, Jensen Huang and AI

Fidelity check: M9, The AI moment and the industry#

Working file. Checks working/maynard-lens/M9-the-ai-moment-and-the-industry.md (version of 26 September 2026) for fidelity to Maynard’s own texts. Line numbers (L) refer to that file. Points about Huang are included only where they change how Maynard is read; the fairness check covers Huang.

Method#

Verdict#

M9 is accurate at the level of wording. Its quotations are exact, its dates are right, and it treats provenance with care. It does not use AI and the Art of Being Human in any form (see issue 14 for three near-misses that it handles correctly). It uses no AI-generated text as evidence, and it weights Maynard & Garbee (2019-08-13) as fully his. Both September 2026 clarifications are reflected faithfully (§2.4, D3, D7, §6 item 3). The INTERNAL section is at the end.

The problems are in how evidence is weighted and labelled, and most of them lean the same way: they bring Maynard closer to Huang, or to a tidier position, than his texts allow. - Two quotations are cut so that they tilt him towards Huang (issue 1). - [Stated] is attached to alignments and applications he has never stated (issue 2). - Five sources are used outside their context (issue 3). - The “AI working as designed” / “narrow aperture” thesis is drawn more narrowly than his risk landscape, and it leaves out his own concessions to the frameworks (issue 4). - Several directly relevant statements of his are missing. The most important is his own description of Huang’s depiction of AI in the Series introduction (issue 5). - D4 settles a tension in his record in one direction only (issue 6).

Most fixes are a relabel, a restored clause or an added sentence. Issues 4 and 5 need a paragraph each.


Ranked issues#

1. HIGH: Two truncated quotations tilt him towards Huang (§5 L188; A4 L109)#

(a) The costs of alarm (§5, last bullet, L188). “Forgone innovation and false alarms have victims (Testimony 2006; FFTF p.163; NN 2014-03 p.160), as Huang’s radiology example shows [Stated].” - Source. NN 2014-03 p.160 praises the RS–RAE report as “a voice of scientific reason and social concern at a time when speculation could have scuppered the nanotechnology enterprise or, worse, led to materials and products that showed a blatant disregard for health and environmental risks.” The map (§2 “It counts both sides”; C7; tension 16) records that he ranks disregard of risk as the worse outcome. M8’s fidelity check (issue 1) raised the same point. - Fix. “…have victims (Testimony 2006 p.52; FFTF p.163) [Stated]. He does not weigh the two equally, though: in 2014 he judged products showing ‘a blatant disregard for health and environmental risks’ a worse outcome than an industry scuppered by speculation (NN 2014-03 p.160) [Stated]. That Huang’s radiology case illustrates the point is [Implied].”

(b) Alignment research (A4, L109). “…and wrote in 2026 that alignment research ‘deserves the attention it’s getting’ (FWB 2026) [Stated].” - Source. FWB 2026 continues: “The technical challenges are real and important. The alignment problem deserves the attention it’s getting. But underneath all of it is a challenge that I suspect matters more than any of the technical ones: figuring out what kind of future we actually want…” - Fix. Quote the “But…” sentence. Also add that 2024-06-20 (“always a social and political endeavor as well as an engineering challenge”, already in §2.4) qualifies the alignment. This matters because A4 is used in the Summary (L20) as agreement “on treating safety spending as serious engineering”.

2. HIGH: [Stated] attached to alignments and applications he has not stated (systemic)#

§8 of the map and M9’s own §2.8 say, rightly, that he has written almost nothing on the July–September events and nothing on Huang beyond the Series introduction. Several labels contradict this: the component quotation is his, but the relation to Huang, the labs or the event is the analysis’s.

Location Text carrying the label Correct label
A2 (L105) promoter–overseer rule “is the same principle at institutional scale [Implied; structural transfer]“ [Inferred, medium]. Huang’s watchdogs are agents monitoring agents inside one firm. Maynard’s principle is institutional separation, and at that scale it cuts against Huang’s “absent of external intervention” (as M9’s own §3.4 row 1 shows). Say that it cuts both ways
A4 (L109) risk innovation, 10% ask, alignment research [Stated] quotations [Stated]; the alignment with Huang’s evaluation compute [Implied, medium]. See issues 1b and 9
A6 (L113) collaboration over “isolationism” and the race-logic criticism [Stated] quotations [Stated]; alignment with Huang on China [Inferred, medium] (see issue 12)
A7 (L115) “counted open-weight support among the Action Plan’s better elements on similar grounds [Stated]“ [Implied, low–medium]; see issue 3c
D2 (L121) “Maynard’s work treats such behaviour as emergent … [Stated]“ components [Stated]; applying 2025-08-31 (a companion-chatbot case) to agentic intrusion is [Implied]; see issue 3d
D4 (L125) “Maynard’s incentive field agrees with the labs’ diagnosis rather than Huang’s … [Stated]“ incentive field [Stated, mixed; older roots 2019-08-13, 2023-11-18, NN 2016-06]; “rather than Huang’s” [Implied]; see issue 6
D5 (L127) “Maynard’s most plausible AI harms come from AI working as designed … [Stated]“ [Implied]; see issue 4
§2.4 (L57) “the 2026 paper keeps capability thresholds while adding a layer [Stated]“ [Stated, mixed]
§3.4 (L143) warn-yet-build “Applies to the labs, not Huang. [Stated]“ [Stated] for the labs (2026-09-15 n.3 names Amodei’s pacing post and the Coxon resignation); “not Huang” [Implied]
§4.3 (L174) “his structural account denies the inference that warnings are ‘deflection’ [Implied, medium-high]“ [Inferred, medium]. He has not addressed whether warnings deflect blame, and his only comment on the warnings (“flummox me”) is closer to Huang’s
§5 (L180) Huang’s objection to human words “fits Maynard’s warnings … [Implied, medium-high]“ [Inferred, low–medium]; see issue 3e
§5 (L182) “the kind of investment he sought for risk research in 2006–08 [Implied, high]“ [Inferred, medium]; see issue 9
Summary (L18) “Maynard’s work reads this as an early-window moment … the questions that matter are…” (unlabelled) mark as the analysis’s reading (“Read through his work, …”)

3. HIGH–MEDIUM: Five sources used outside their context#

(a) 2025-05-04 (§2.4 L57; D3 L123). Cited [Stated] under “tests have limits”, and paraphrased in D3 as “simulated environments leave real-world risk unclear”. - Source. “Simulated environments” is Kasirzadeh and Gabriel’s category of operating environment for agents (beside mediated and physical). His sentence is a doubt about their efficacy scale: “I’m not fully convinced that the proposed efficacy scale has it right yet, as it’s unclear how the risks of an AI working within a simulated environment (the ‘safest’ type of environment) might be realized”. It says nothing about testing, and the D3 paraphrase reverses his direction (he is unsure how risk arises in a simulated environment, not how simulation predicts the real world). M8’s check (issue 6) found the same misuse. - Fix. Delete it from §2.4 and D3. Testimony 2007 p.21, NN 2015-06 p.483 and the September 2026 clarification carry the point. If it is kept, gloss it accurately, “he doubted that a framework rating simulated environments ‘safest’ had captured how risk arises there”, and label it [Inferred, low] for the July incident (a “supposedly isolated sandbox”, 2026-09-24). That reading is at least on topic.

(b) 2023-11-18, “a chance to take a breath (a pause even)” (§3.1 L92; §5 L185). Offered as “Costly unilateral action is what he has asked for”, and in §5 as pauses “which his work asked for”. - Source. He speculates that the OpenAI board crisis “may be a good thing — a reset from the hype of the past year, and a chance to take a breath (a pause even) and rethink and recalibrate the relationship between AI and society”. It is a hope that disruption might give society a pause, not a request for firms to pause at their own cost. His record on pauses is mixed: - he declined the 2023 pause letter (2023-04-04); - he called for “pausing — or even rethinking” emotion-exploiting companion bots (2024-10-27); - he said in 2026, “We can’t pause it” (2026-09-24 [mixed]). - Better evidence. His paper treats Anthropic’s move from an “unconditional” pause to one “conditioned on what competitors do” as a softening (2026-07-16 [mixed], arXiv p.8). That implies he values unconditional pause commitments. The 2025-08-31 call for “working harder on safety checks and protocols before releases” can stay. - Fix. Replace “what he has asked for” with “consistent with his paper’s treatment of unconditional pause commitments as the stronger form (2026-07-16 [mixed]) and his call for more safety work before release (2025-08-31)”. Cite 2024-10-27 as his one explicit call to pause a class of products, note the 2023 decline and “We can’t pause it”, and label [Implied, medium].

(c) 2025-07-23, open weights (§2.7 L74; §3.1 L97; A7 L115). The text says he “listed … support for open-weight models among its better elements” and speaks of “his” “worry about ‘corporate control’ of closed models”. - Source. “On the other hand, the Action Plan promotes open-source and open-weight models — something that will be welcomed by many who worry about the corporate control…”. He then adds that researchers will applaud it, and that these are open models “with strings attached”. The worry is attributed to “many”; the framing (“On the other hand”) is mildly positive, but it is reportorial. - Fix. “He noted, in a positive aside, that its open-weight support ‘will be welcomed by many who worry about the corporate control’ of closed models, adding ‘strings attached’ (2025-07-23) [Stated]; whether he shares the worry is [Implied, medium].” In the §3.1 open-weights row, attribute “corporate control” to those he describes. For balance, §2.7 should add that he gave input to Howard’s pro-openness paper (“myself included”, 2023-07-12).

(d) 2025-08-31, “better-managed, but probably not eliminated entirely” (§2.5 L64; D2 L121; §3.1 L92). Presented generically (“Harmful behaviour that emerges from a model’s properties…”) and applied to agentic intrusion as [Stated]. - Source. The sentence is about “The alleged behavior of ChatGPT that led to Adam Raine’s death”, an emergent property of a conversational model, contrasted with apps “intentionally designed” to exploit cognitive biases. - Fix. Name the context in §2.5. Label the transfer to July’s agents [Implied, medium]; it is a reasonable structural transfer (emergent behaviour managed, not eliminated), but it is not his statement about agents. Stronger [Stated] support for D2 exists: see issue 5b and 5f.

(e) 2024-05-15, “our anthropomorphizing cognitive biases” (§5 L180). Used to show that Huang’s objection to human words for software processes “fits” Maynard. - Source. The phrase defines “hyper-anthropomorphism — a concerted effort to create AI’s that are intentionally designed to engage our anthropomorphizing cognitive biases”, i.e. products “designed to make us fall a little in love with them”. It concerns designed intimacy, not how critics describe software. Huang’s target (“Spawn, create, kill…”, “A collection of people want to make the software more than it is”) is discourse. Maynard also keeps goal-directed language himself (issue 7). M8’s check (issue 11) found the same problem. - Fix. Narrow it to “both resist mystification”, note that his concern is designed anthropomorphism and its effects on users, which Huang’s framing does not address, and relabel [Inferred, low–medium].

4. HIGH–MEDIUM: “Harm from AI working as designed” and “narrow aperture” drawn more narrowly than his record, and his concessions to the frameworks omitted (D5 L127; §3.4 L140; §4.3 L171; Summary L20, L22)#

Text. D5: “Maynard’s most plausible AI harms come from AI working as designed, through fluency and relationship (Trojan 2026 p.1; Harness 2026 p.8) … [Stated]”. §3.4 row 2 and §4.3 build the main proxy finding on this.

Sources. - Trojan 2026 p.1 is a scope statement, not a ranking: “the analysis focuses on AI systems designed to be genuinely useful; the distinct challenges posed by intentional use of AI for manipulation … while important, fall outside the present scope.” His [Stated] plausibility ranking (FFTF p.159) sets manipulation above superintelligence. It does not set as-designed harm above failure. - His landscape is plural and includes failure-type risks. The week before the interview he reaffirmed as “amongst the top longer term (and more insidious) risks associated with frontier models” value misalignment, “Machines that alter their own instructions”, “smart-dumb decisions” and lethal autonomous weapons. He named cybersecurity and “governance of frontier AI models and systems” among the risks that have risen (2026-09-15). His own reading of the July incident is a loss-of-control reading (2026-09-24 [mixed]). The Series introduction’s reservation is about a missing “broader landscape of emergent AI characteristics, capabilities, threats, risks, and benefits”, which is broader than the cognitive thread. - His paper concedes the frameworks’ logic. “To be fair to the frameworks’ designers, there is a case for focusing on a narrow but deep risk layer … Concentrating limited safety resources on risks perceived to have the highest levels of severity makes sense; a framework that tried to manage sixteen hundred risks would end up managing none of them. And the frameworks’ architects have, sensibly, never claimed completeness.” Capability thresholds are “reasonable engineering judgment”, and “Severity floors make sense as triage” (2026-07-16 [mixed], arXiv pp.5, 7–8). None of this appears in D5, §3.4 or §5.

Fix. - D5: “The AI risk he has developed most fully, a threat to how people think and trust, operates, on his 2026 account, through AI working as intended (Trojan 2026 p.1; Harness 2026 p.8) and accumulates below catastrophe thresholds (2026-07-16 [mixed]) [Stated]. A containment-and-release model does not see it [Implied, high]. His landscape also includes failure and agency risks (2026-09-15), on which Huang’s containment model does engage [Stated].” - §3.4 row 2 and §4.3: add his concession (narrow-but-deep as defensible triage) as [Stated, mixed]. Say that his objection is to the missing second layer, not to the narrowness of the first (“a single safety layer is currently being asked to effectively stand in for two”, arXiv p.18). - §5: add a bullet, “Capability thresholds as defensible triage”, [Stated, mixed].

5. MEDIUM–HIGH: Directly relevant statements of his are missing#

Adding these would turn several [Implied] points into [Stated] ones, or balance them.

6. MEDIUM: D4 and the Summary settle a tension in his record in one direction (L20, L125, L142)#

Text. D4 says his incentive field “agrees with the labs’ diagnosis rather than Huang’s [Stated]”. The Summary lists “whether competition drives drift” as a divergence from Huang.

Source. His only direct comment on the September pacing calls is 2026-09-15 n.3. It names the Coxon resignation and “calls from Dario Amodei and others in the industry for more caution”, and says: “it does flummox me a little as to why the people developing AI are the ones both saying they should go slower, and not doing so”. In substance that is close to Huang’s “That strikes me as odd” [53:36]. Against it stands his structural account: the incentive field ([mixed]; the map lists the “incentive field” analysis among concepts that may have originated with Fable) and its secure roots (2019-08-13; 2023-11-18; NN 2016-06 p.491; 2024-07-13).

Fix. - D4: present both halves as his and let neither absorb the other. His most recent direct reaction echoes Huang’s impatience [Stated]. His structural account explains the drift without bad faith and prescribes common rules rather than courage [Stated, mixed; roots Stated]. The divergence from Huang is over the remedy and the explanation, not over whether the labs’ position is odd [Implied]. - Summary (L20): “diverges … on whether competition, rather than courage, explains the drift and what would fix it”.

7. MEDIUM: A1 overstates his “deflation” and misplaces Huang’s remark (L103; Summary L20)#

8. MEDIUM: The Summary misstates the [mixed] paper (L18)#

Text. “…with some set-aside risks reappearing only where law compels them [mixed]”.

Source. The paper documents a voluntary return: “Google DeepMind … moved the other way, adding an avowedly exploratory harmful-manipulation domain to its own framework in 2025”, which shows exclusion is “at least in some cases — a choice rather than a necessity”. It also documents Anthropic’s manipulation tiers, “made before any enforcement”. M9’s own §2.3 reports the first correctly.

Fix. “…reappearing mainly where law requires them, with Google DeepMind’s voluntary addition as the exception [mixed]”.

9. MEDIUM: His 2006–08 funding ask is equated with firm-internal evaluation compute (A4 L109; §5 L182)#

Text. A4: “asked Congress for a tenth of nanotechnology research spending for risk research (Testimony 2006–2008)”. §5: the tenfold prediction and the “flip” to evaluation “are the kind of investment he sought for risk research in 2006–08 [Implied, high]”.

Source. - The 10% ask is in Testimony 2007 and 2008. In 2006 he asked for a fixed sum, as M8’s check also found. - The ask was for “strategic risk-related research” directed by federal agencies with an oversight mandate, partly because industry “has an economic incentive to sell products” and its findings “might be considered suspect … if not supported by independent studies” (PEN 2006 pp.4–5, 32). He also proposed bodies funded jointly but operating “independently of the funders” (WEF 2008). - Firm-internal evaluation compute meets the scale condition, not the independence condition. - Huang’s tenfold figure is a forecast (“I wouldn’t be surprised if…”), not a demand.

Fix. Date the ask “Testimony 2007–08”. Relabel §5 [Inferred, medium], adding: “His 2006–08 ask was for independent, strategically directed research, so he would value the scale of the shift more than its location inside the firms.” A4 should say that it pairs a forecast with a demand.

10. MEDIUM: §6 items 4 and 5 present the analysis’s instruments as his direction (L200, L202)#

11. MEDIUM: Proportion and provenance: the [mixed] paper carries the headline finding, and single-source items are not flagged (L7–8, L22, L140, L160, L164; §7 L221)#

Fix. In §2.3 and §4.2, cite these first and the paper second. That anchors “aperture” in his own long-standing idea that what gets measured decides what gets managed. - Unflagged single-source items. The conventions (L8) promise that neither [mixed] text “carries a position alone unless marked ‘single source’”. Unflagged exceptions: - the four filters (§4.1 “[Inferred, high]”; possibly Fable-originated, per map §1); - the safety differential and aperture log (§4.2, §6 items 1–2); - “irrelevant to this conversation” (A1; lecture only); - the race-logic line (A6; §3.1 China row; lecture only); - “knowing that it’s speculation, not reality” (§6 item 8; corroborated by “These are explorations, not findings”, HNS 2026, which should be cited).

Flag each, or add the corroboration. - Balance of his thinking. Being human, the formative and cognitive thread, and justice appear only in passing (D5, D8, the Australia row). That is defensible for an industry-facing dimension, but §7’s list of what the reading “also rests on” should not imply more than the text delivers.

12. MEDIUM–LOW: A6 reads his race-logic criticism as support for Huang on China (L113, L98)#

Source. - The lecture line targets speed under ignorance, whoever argues for it: “the companies will tell you — and governments will tell you too — that we have to go as fast as possible with it, even though we don’t know what it is that’s happening, because if we don’t go fast, somebody else will” (2026-09-24 [mixed]). That applies to Huang’s own “We’re racing as fast as we can” (HA §8.1 T13) as much as to export controls. - The 2025-07-23 passage concerns a US-centric “dominance” vision, with collaboration preferred to “isolationism”. - Earlier he named a US–China AI “arms race” as a concern (2023-07-25), and treated arms-race logic as a limit on responsible innovation (2025-04-06, his framing only; the report is o1-pro’s).

Fix. Keep the alignment on collaboration versus zero-sum framing [Implied, medium]. Add that his race-logic criticism cuts against Huang’s speed rhetoric too [Implied, medium]. Do not use 2025-04-24 (us-and-china-vie-for-ai-k-12-leadership) as evidence: the concept index records it as “Developed by AI, checked by humans”.

13. LOW: Citation, wording and presentation slips#

Location Issue Fix
§2.6 (L70) “waking up … first people to notice them” is in the main text, not n.3; only “flummox” is n.3 “(2026-09-15; n.3 for ‘flummox’)”
§3.4 (L143); INTERNAL “first to notice” in quotation marks; his words are “acting as if they’re the first people to notice them” quote exactly or drop the marks
§2.1 (L30) Hansen et al. 2008 “pp.444–447” for one quotation p.447 (co-written, as marked)
A2 (L105) PEN 2006 p.32 is industry’s “economic incentive”; the promoter–overseer rule is PEN 2006 pp.4–5 and 17, Testimony 2007 p.30, Hansen et al. 2008 p.446 cite these
A2 (L105) “six months earlier” than what? 2026-01-31 is about 5½ months before the incident and 8 months before the interview “some five months before the incident”
A5 (L111) “His reversibility test (2025-03-02) marks where this stops applying (D3)”; D3 is about tests point to §6 item 7; also note that the 2024-02-25 context is Gemini’s image generator, which he called “more of a ‘gotcha’ moment … than a dangerous flaw”
D6 (L129) “leaving these questions to innovators” drops “solely” and “experts” (FFTF p.288: “leave to ‘experts’ … leave solely to people like scientists, innovators, and politicians”) restore “solely”
§3.1 (L96) “whose future” in quotation marks, cited to 2019-03-31; the phrase is not in that post (its aphorism is “the future is designed by the powerful”) drop the marks or quote the aphorism
§6 item 10 (L212) “universities as conveners”; his words are that universities could “play leadership roles” and bring insights “to the table” (2026-08-30) use his wording
§2.5 (L62) “motive defined … to avoid anthropomorphism” “defined in reply to a charge of anthropomorphism, while defending the term” (issue 7)
§2.8, Conventions (L7) Series introduction cited as published in The Future of Being Human on 27 September 2026. On 26 September it is draft 3, with “[add url]” placeholders and typos cite as “forthcoming” or confirm the wording after publication (INTERNAL Q9)
Header (L3) “for his review” “and reviewed by him”, once reviewed (publication-context wording)
Conventions (L9) file path “working/synthesis/” in the public text name the document, not the path

14. LOW (no change needed): excluded text near cited passages#

M9 is clean, but three cited sources sit beside AI and the Art of Being Human. Future edits should keep it that way. - 2026-07-16, the Vaughan paragraph. The paragraph ends by equating Vaughan’s mechanism with “values drift” from the book (“the small yes that makes the next yes easier”, note 29). M9 uses only the Vaughan part (§2.2, §3.1, INTERNAL Q2). Do not add “values drift” or that quotation. - 2026-05-21. “the boat has already left the harbor” is in the sentence that follows a mention of the book. The sentence is his own prose; cite the post only, as M9 does. - 2026-05-10. Note 10 names the book. M9 uses only the main text (“safety message first”; “near-impossible”).


Checked and correct (selection)#


INTERNAL (not for publication)#

Questions for Maynard arising from this check (the M9 questions are not repeated): 1. Did 2023-11-18’s “a pause even” express a wish that firms would pause, or only a hope that the turmoil would give society room to recalibrate? 2. In 2025-07-23, was the open-weights aside (“welcomed by many who worry about the corporate control”) your own view? 3. May the Series introduction’s sentence on AI “as it is depicted by Huang” be quoted in the public documents before it is published?