Late Lessons, Jensen Huang and AI

Fidelity review: 05-maynard-risk-and-ai-map.md#

Reviewed 26 September 2026 against working/maynard/corpus/ (391 posts), working/maynard/manifest.json and working/maynard/book/FFTF-*.txt. Line numbers (L) refer to 05-maynard-risk-and-ai-map.md.

Scope and method#

Overall. The map is accurate at the level of wording. Every date–slug pair matches the manifest. No quotation was found to be fabricated. No quotation from a known AI-generated block (Perplexity takeaways, the o1-pro report, ChatGPT transcripts, the Manus report) or from a guest post is passed off as his. Allenby, Lobo, Boorsma and Garbee are correctly flagged. The problems lie in provenance framing for 2026, in a few quotes torn from their hedges, in several labels that are the mapmaker’s rather than his, and in a small number of dating and citation slips.


Ranked issues#

1. The frontier-AI paper’s analysis is treated as a shift in his own thinking (HIGH)#

Where - L38 (provenance note) - L132 (“→ the incentive field”) - L256 (commitment 9: “By 2026 the account is structural”) - L385 (concept row) - L503 (T1: “finally at frontier-AI governance”) - L535 (T5: “Two shifts stand out”) - L607 (§7 What changes #6) - L647 (§8.4: “He acknowledges this only in the 2026 paper”) - L709 (lens 9)

Problem. The provenance note names only the “four filters”, “safety differential”, register and aperture log as possibly model-originated. The map then treats “sincerity almost always operates inside an incentive field” as the second of two major shifts in his governance thinking, and as a change in how he explains developers. It treats the “conversion channels … not equally open to everyone” passage as his own self-acknowledgement. The phrase “incentive field” occurs in no other post in the corpus.

Evidence - In 2026-07-04 just-how-good-is-anthropics-fable-as-a-research-assistant he says Fable applied his risk-innovation work to frontier AI “in a way that hadn’t previously occurred to me”, and calls “the framing and analysis” “genuinely novel”. - The same post attributes the drafting to Claude: “drafted by Claude (Anthropic’s Fable 5 …) under Andrew Maynard’s direction”. - In 2026-07-19 he writes that “the ideas, analysis and insights that Fable generated remain intact”. - The only earlier structural account in his own voice is the 2024-07-13 economic gradient (ai-choice-engines-sunstein).

Fix - Add 2026-07-04 to the §1 provenance note. - State that the frontier-AI application, including the incentive-field analysis and the unequal conversion channels, may be Fable’s. Only the threat-to-value frame, orphan risks and the Garbee lesson are securely his. - In §7 #6 and T5, rewrite along these lines: “a structural account appears in the Fable-assisted paper [mixed]; his own precursor is the 2024 economic gradient”. - In §8.4, replace “He acknowledges” with “The Fable-assisted paper acknowledges [mixed]”. - Mark lens 9 as drawing partly on mixed text.

2. The 2026-01-17 post is misdescribed, and a larger AI contribution is omitted (MEDIUM-HIGH)#

Where: L39 (and the [mixed] tags at L469 and L665)

Problem - The map calls the post “Claude-drafted”. The post itself is his own first-person account. What Claude drafted is the arXiv paper the post describes. - The map records only that “honest non-signals” came from Claude. - It omits his fuller credit, which bears directly on the “cognitive Trojan horse” thread that the map ranks as the 2026 culmination of his most continuous AI thread (L85, L468).

Evidence (2026-01-17) - “the concept of honest non-signals came from Claude, as did the development and refinement of the various mechanisms by which conversational AI might slip by our epistemic vigilance mechanisms.” - By contrast, the 2026-01-10 post was “primarily based on my own thinking”, with “some initial brainstorming with Anthropic’s Claude”.

Fix - Reword L39: “His own account of writing a Claude-drafted arXiv paper. He credits Claude with ‘honest non-signals’ and with developing the mechanisms of vigilance bypass.” - Anchor the cognitive-Trojan-horse thesis on 2026-01-10 (his own prose, with fluency, attractiveness, speed and volume, and the intelligent user trap). Treat the paper-level mechanism account as mixed. - His own quotes in the reflection (for example “how do I know I’m not an unwitting victim here?”) need no [mixed] tag.

3. “One of the scariest things I’ve ever seen” is quoted without its counterweight (MEDIUM-HIGH)#

Where: L89 (§2: what AI “is for him”); L519 (T3, 2026)

Problem. Used on its own, the quote overstates his alarm. In the source it is one half of a deliberately balanced sentence.

Evidence (2026-09-24) - “I’m stuck between thinking this is one of the scariest things I’ve ever seen … and, at the same time, that the potential is profound.” - It is preceded by “I’m not a strong AI optimist or advocate”. - It is followed by “So I’m neither an AI optimist nor an AI pessimist.”

Fix: Quote it as “stuck between” fear and “profound” potential, or pair it with the “neither optimist nor pessimist” line in both places.

4. “Formation” is ranked Core from 2024 on thin and misread evidence (MEDIUM)#

Where: L472 (row: “2024-10-20; 2026-06-10; 2026-09-24 [mixed] | Core (from 2024)”); L85, L302–308, L586

Problem. The cited sources do not support the rating.

Evidence - 2024-10-20 is a thought experiment about how people learn social agency. It never speaks of formation. - 2026-06-10’s “the formation of those outputs” refers to AI producing learning outputs, not to human formation. That passage also sits where he agrees with “Claude’s conclusions”. - The only clear statement is the lecture: “Language is formative … a technology that was actively taking part in the formation process” (2026-09-24 [mixed]). He adds that the claim is “somewhat controversial”. - The only other formation-of-persons passage in the corpus is the 2026-06-14 PhD note (“a process of formation”), which is excluded as mostly AI-written.

Fix - Drop 2026-06-10. - Re-rate as “Emerging (2026; lecture [mixed])”. - Give 2024-01-01 (intrinsic technologies that change “what we are”) and 2026-07-19 (“train us to think like them”) as his precursors.

5. The mapmaker’s coinages are presented as his concepts (MEDIUM)#

Where: “reverse formation” at L471, L527, L586, L655; “validation gap” at L473; “second-order evolutionary mismatch” at L468

Problem. None of these terms occurs anywhere in the corpus. “Reverse formation” is even listed among his “signature new ideas” for 2026 (L586).

Evidence - Corpus grep finds zero hits for all three terms. - His own wordings are “the AIs we have trained to ‘think’ like us are now beginning to train us to think like them” (2026-07-19), and AI generating knowledge “faster than we are currently capable of validating and even understanding” (2026-06-12, postscript).

Fix: Mark these as mapmaker labels (for example “my label:”), or replace them with his wording.

6. The “Signalled changes of mind” table mixes inferred shifts with signalled ones, with one misquote and one misread (MEDIUM)#

Where: L612–629

Problem and evidence - 2018 “Helped build ‘brand nano’” (L617). - The source spells it “brand-nano” (2018-02-21). “brand nanotechnology” is in FFTF p.217. - He never says he helped build it, and no reversal is signalled. The post is a sceptical insider view: “my BS monitor also gets a little twitchy”. - 2026 “‘Stochastic parrot’ language” (L627). - In 2025-02-23 he calls Evo 2, a DNA model, “a DNA-based stochastic parrot”, with approval of its power. That is not a claim about LLMs. - In 2025-01-05 he endorses Melanie Mitchell’s view that models are “pushing far beyond critiques that … AIs are simply ‘stochastic parrots’”. - So there is no reversal to signal. - 2015→2016 (L616) and 2025 exponentials (L625). These are the mapmaker’s inferences. He did not signal them.

Fix - Split the table into “Signalled by him” and “Inferred”. - Delete the stochastic-parrot row, or recast it as continuity. - Correct the quote to “brand-nano” and describe the row as “sceptical reassessment of his own field”, not “helped build”.

7. AGI is not “irrelevant” to his risk map (MEDIUM)#

Where: L436 (“by 2026 AGI ‘irrelevant’ to his risk map”)

Evidence (2026-09-24 [mixed]) - “I am not talking about AGI … superintelligence … consciousness. All of those might happen. But I think they’re irrelevant to this conversation.” - The conversation in question is loss of control without AGI. - §8.14 (L667) quotes this correctly.

Fix: “by 2026 AGI set aside as ‘irrelevant to this conversation’ about loss of control (2026-09-24 [mixed])”.

8. The 2026-08-30 quotes are spliced, and “failing” overstates his view (MEDIUM-LOW)#

Where: L659 (§8.10)

Evidence - “Sadly, this has been my experience so far” is footnote 7. It is attached to his fear that universities “may not be up to the task” and will treat AI as a minor challenge. It ends “But there’s always hope.” - The “guardians of the past” sentence has footnote 6 instead, which softens it: “This, I realize, probably comes across as a little harsh.” - The map sets the two side by side and concludes that he describes universities “as failing”.

Fix: Quote footnote 7 with its real anchor and its closing line. Replace “describes as failing” with “fears may not be up to the task”.

9. Pippard’s ladder and tipping points are dated to 2023, against the map’s own convention (MEDIUM-LOW)#

Where - L17 (lists 2023-05-04 as a Future Rising excerpt) - L410 (“First / key dates: 2023-05-04; 2024-08-18”) - L543 (T6: “Pippard’s ladder … (2023)”) - L252 (the 2023-05-04 quote dated 2023)

Evidence - 2023-05-04 is not a Future Rising excerpt. It says “much of this essay draws on” an early 2018 draft of the FFTF climate chapter, and that the idea “did resurface in my 2020 book Future Rising”. - 2024-08-18 says “I first wrote about Pippard’s ladder in my book Future Rising”. - 2024-05-19 and 2020-10-30 are further Future Rising posts that the list omits.

Fix - Date the tipping points / Pippard idea as 2018 draft → 2020 (Future Rising) → 2023 revival. - Correct the §1 list of Future Rising posts.

10. Citation slips in the §5.8 rows (LOW-MEDIUM)#

Where: L470, L471

Evidence - L470 (cognitive surrender / “easy button”). The row cites 2024-01-07; 2026-05-10; 2026-05-21. - “Easy button” occurs only in 2026-09-24 [mixed]. - “Cognitive surrender” occurs in 2026-05-21 (credited to Shaw and Nave), 2026-07-16 and 2026-09-24, but not in 2026-05-10. - L471 (constitutive resonance). The row cites 2026-02-22; 2026-03-08; 2026-07-19. The term appears only in 2026-05-21, which refers to his SSRN preprint.

Fix: Add 2026-09-24 [mixed] to L470 and remove 2026-05-10 from the “cognitive surrender” part of that row. Add 2026-05-21 to L471.

11. The orphan-risk count is wrong on timespan (LOW)#

Where: L503 (“16 posts in eleven years”)

Evidence: The 16 posts run from 2018-12-13 to 2026-08-02, which is under eight years.

Fix: “16 posts, 2018–2026”.

12. Hedges dropped from source quotations (LOW)#

Where and evidence - L551 and L796, “childish irresponsibility”. The map attributes it to “Altman’s handling”. The source says “a reality that sometimes seems childish irresponsibility” and refers to “AI companies like OpenAI” (2024-05-21). - L604 and L643, “plans ‘just on the off chance’”. The source says it “does force the question of how we might think about responsible innovation and AI, just on the off chance …” (2025-04-06). That is thinking, not planning. - L296, manipulation over superintelligence reaffirmed “every year since 2023”. - The explicit comparison is restated only in the 2023-04-16 repost and in 2025-07-06 fn5. - 2026-09-15 lists heuristic manipulation among the ten risks without ranking it.

Fix: Restore “sometimes seems” and “AI companies like OpenAI”. Change “plans” to “thinks about”. Soften to “reaffirmed repeatedly (2023, 2025)”.

13. §8.2 overstates how strict his 2018 plausibility rule was (LOW)#

Where: L643

Evidence: The same page the map cites (FFTF p.281) warns that Occam’s Razor should “never be considered as more than an aid”. It adds that it “leaves the door open to more complex, more fanciful possibilities being plausible”.

Fix: Note that the 2018 text already anticipates the tension, so the change is one of emphasis, not a reversal.

14. Minor accuracy and citation points (LOW)#


Checks that passed (selection)#

Book (FFTF) - p.22 “less and less patience…” - p.23 values list, and “new wine … old wineskins” - p.38 “their version of ‘responsible’” - pp.118–120: thirteen years at HSE and NIOSH; grandfather’s pneumoconiosis (1977); “uncertainty that suited the mine owners” - p.121 “first tier” - p.150 “continuing duty of care” - p.159 “far more plausible, and far scarier” - p.161 the rule-bending all-nighter - pp.162, 166, 167: permissionless-innovation quotes - p.168 “AI wasn’t even on my radar” (2008) - pp.170–171: “something of an agnostic”, “I freely admit…”, “currently scientifically implausible”, “a term of convenience” - pp.174, 205, 222, 227, 246, 249, 281, 288, 290: all confirmed

Posts, 2014–2023 - 2016-01-11: “parallel innovation…”; threat to value as founding frame - 2018-12-13: definitions, “extends conventional thinking”, “somewhat subjective” - 2018-05-12: the ten-risk list - 2019-03-05: “No exposure means no risk…”; “an algorithm is not a chemical” - 2019-04-15: “worth little without…”; “smoke-and-mirrors” - 2020-07-30: COMEST “a sound philosophy”; “more to risk than probabilities” - 2020-10-15: “somewhat naïve” (Neuralink) - 2020-11-05: outmoded risk ideas as a risk - 2023-04-04: pause letter not signed, with “potentially existential proportions” (I do); “world congress” - 2023-05-15: “everyone has the right…”; “it’s complicated, leave it to us” - 2023-05-31: title “why I didn’t sign it”; catastrophe as mass loss of value; “shavings off the tip” - 2023-08-14: “profoundly effective catalyst” and its caveat. His prose, not ChatGPT’s. - 2023-09-20: “would be to diminish myself” - 2023-11-18: “(a pause even)” - 2023-11-21: value vs values, “agnostic to particular worldviews”. His prose, before the ChatGPT role-play. - 2023-11-26: “no cause, no risk”; the addendum is his own; “zero exposure — as in no AI”; “understanding-vacuum”

Posts, 2024–2025 - 2024-01-01: extrinsic/intrinsic technologies; “what we are”; base code “without our agreement” - 2024-01-18: welcomed the ASU–OpenAI partnership - 2024-02-25: signed the deepfake letter (criminalisation, developer liability); “far less sure” - 2024-03-31: oxygen analogy; “technology apologetics” - 2024-05-15: hyper-anthropomorphism - 2024-06-20: “social construct, not a technological one”; who decides what “safe” means; zero risk only without change - 2024-09-01: “who decides what is good for society?” - 2024-09-22: “hard not to trust, and yet are not trustworthy”. His prose, not the AI podcast text. - 2024-10-08: “a generator of ideas, not an understander…” - 2024-10-27: “random and unpredictable”; “pausing — or even rethinking” - 2025-03-02: Musk “naive”; DOGE; reversibility footnote; the FFTF excerpt republished - 2025-03-15: “categorical error” (footnote) - 2025-04-06: all quotes come from his framing, before the o1-pro report; “futile” is hedged - 2025-07-06 fn5: “I wrote about this back in 2018” - 2025-08-10: “AI denial”; dismissing AI over hallucinations “naive” - 2025-08-31: regulate designed exploitation; flood metaphor; “some degree of vulnerability” - 2025-11-09: iceberg; literacy classes “risk becoming performative” - 2025-11-19: “read the tea leaves wrong”

Posts, 2026 - 2026-01-10: fluency, attractiveness, speed/volume, intelligent user trap; “only a small chance”; “admittedly limited analysis” - 2026-01-22: “defies analogy”; “Rather, they are different” - 2026-02-08: “suckered by Claude” - 2026-04-11: AGI “rather ill-defined” - 2026-04-26: “a relational technology”; “character constancy” - 2026-05-10: five rules; “safety message first”; “near-impossible” at his institution; “Luddite” - 2026-05-17: “integral” - 2026-05-21: cognitive surrender credited to Shaw and Nave; “potentially dangerous”; “shaking things up” - 2026-07-19: “blown away” → “superficially profound yet substantively hollow”; “remain intact” - 2026-08-30: “followers and users of the technology” - 2026-09-15: ten risks “still surprisingly relevant”; the list of additions; all footnotes - 2026-09-24: the map’s description of this post’s provenance (intro and first part of postscript his; Opus 5.5 draft; his line edit) is accurate; “We can’t pause it” is correctly tied to “may be a flawed assumption”

Attribution, citations and exclusions - Guest and co-written posts are correctly identified: Allenby (2023-08-16, 2023-10-29), Lobo (2023-09-11), Boorsma (2019-11-19), Garbee (2019-08-13). - All date–slug pairs match the manifest. - No Modem Futura post is used as evidence.