Late Lessons, Jensen Huang and AI
Markdown version

B28 notes: 2025-11-24 to 2026-01-25 (12 posts)#

Reading notes on Andrew Maynard’s Substack posts in batch B28. Only his own prose counts as evidence. Quotes are exact, including his original typos, curly punctuation and italics (shown as asterisks).

Batch context: late November 2025 to late January 2026. The batch opens with four posts serialising his short story Letters from the Department of Intellectual Craft: Parts 1-3 and a closing essay (the Postscript). The story is a chapter in the forthcoming volume Academic Cultures: Perspectives from the Future, edited by Michael M. Crow and William Dabars (Johns Hopkins University Press, 2026). Its Preamble (2025-11-23) belongs to batch B27. Next come a post on the White House’s Genesis Mission and universities, a hands-on test of OpenAI’s custom GPTs, a holiday piece on “foveated reality” and a New Year reading list. January 2026 then brings a tight, connected cluster: - the “cognitive Trojan Horse” essay, which grew out of his OEB25 keynote in Berlin in December 2025; - the AI-assisted arXiv paper that followed from it, and his account of how it was made; - his response to Anthropic’s new constitution for Claude; - a joke “paper” about AI Use statements.

The 2025-12-21 foveated-reality post was prompted by a Modem Futura podcast episode. Per the user’s instruction the podcast itself is not considered. Only his written essay is noted, in one line.

Relevance summary:

Date Slug Relevance
2025-11-24 part-1-letters-from-the-department-of-intellectual-craft medium (fiction)
2025-11-25 part-2-of-letters-from-the-department-of-intellectual-craft medium (fiction)
2025-11-26 part-3-of-letters-from-the-department-of-intellectual-craft medium (fiction)
2025-11-30 postscript-letters-from-the-department-of-intellectual-craft high
2025-12-07 universities-genesis-mission medium
2025-12-14 revisiting-custom-gpts medium
2025-12-21 are-we-living-in-a-foveated-reality low
2026-01-01 five-voices-five-pieces-2026 low
2026-01-10 is-ai-a-cognitive-trojan-horse high
2026-01-17 i-cracked-and-wrote-an-academic-paper high
2026-01-22 think-you-know-ai-think-again high
2026-01-25 can-modern-scholarship-escape-ai low

HIGH#

2025-11-30 — postscript-letters-from-the-department-of-intellectual-craft — “Postscript: Letters from the Department of Intellectual Craft”#

Subtitle: “Plausible AI trajectories and their potential implications to academia: a closing reflection on Letters from the Department of Intellectual Craft”.

Provenance. An essay in his own prose. It is the closing section of the book chapter, reproduced with the publisher’s permission. The footnotes give access dates of 9-10 May 2025, so it was probably drafted in May 2025 and published here in November 2025. He cites Dario Amodei, AI 2027 (Kokotajlo et al.), Mark Daley, McNutt and Crow, C. Wright Mills, Crow and Dabars, and Julie Schumacher. He describes how the story was made. The back-story was “developed through working closely with OpenAI’s deep research model”, which produced over a hundred pages of fictional history, published separately by the Future of Being Human Initiative as “The Legacy of Early 21st-Century Academia: A Foresight Report from 2100” (not in this corpus). The letters themselves were “very intentionally developed and written without the aid of AI (apart from occasional grammar and style checking).” The header image is Midjourney/Photoshop.

Argument in his terms. - Forecasting AI’s impact on academia 75 years out is “a near-impossible task”. In 2100, universities might look “remarkably similar” to today, or “long ceased to exist”. AI is “one of those profoundly disruptive technologies that has the ability to confound even the most prescient of futures-forecasters.” - Three scenarios set the boundaries. 1. Amodei’s AI-driven “compressed twenty-first century”. Academics might then be “relegated to managing the vastly more powerful intellects of intelligent machines”. 2. The more extreme AI 2027 scenario of recursive AI development leading to “superintelligent” AI researchers. He calls it implausible-sounding (“Implausible as this scenario sounds”), but says it would force a “radical rethink of academia within the next decade”. 3. A plateau, in which AI is assimilated “much as electricity, the calculator, or the internet were”. Academia would then still be dominated by “very human aspirations, ambitions, behaviors, and egos.” - His critique of scenario thinking. Scenarios “tend to focus on artificial intelligence as something that happens to society”. But AI is “arguably the first technology” able to emulate and even replicate uniquely human attributes, including intellect. So “the very essence of who we are is intertwined with its development.” He is fairly bold about capability: AI models are “well on the way to progressing from emulating PhD-level intelligence to exhibiting genius-level intellect”. “If and when” that happens, academics may have to concede that AI “is capable of achieving more than the vast majority of academics”. - A double crisis for academia. - A “crisis of identity and purpose”: intellectual elites face, “For the first time in human history”, machines that think and reason deeper and faster. - A “crisis of abundance”. Following Daley, universities are “built on a bedrock of scarcity” of cognitive capacity. If anyone with an internet connection gets more cognitive capacity than a university can offer, “at a fraction of the cost”, the “public value of the academy” will have to be rethought. - Against the purely utilitarian reading. A utilitarian future could leave academic culture “diminished to the point of irrelevance”. But that rests on “a very narrow understanding of the value that academics bring”. Hidden value may lie “under the carapace of bureaucratic performance indicators”, and AI might reveal it. A “future fusion of human intellect with artificial intelligence” might even “transcend utilitarianism and human-exceptionalism in academic culture.” - Intellectual craft. C. Wright Mills (1959) described intellectual work as craft: a process and almost a lifestyle, “neither prescriptive or outputs-driven”. The idea helps “tease out the “secret sauce” of the academic (if indeed such a thing exists) as distinct from mere productivity.” Crow and Dabars call small-scale craft production of knowledge a pre-industrial “boutique” strategy, but grant that artisanal modes “may remain essential to discovery, creativity, and innovation”. He links this to his own “artisanal intellectual” (February 2025) and to “slow scholarship”. In the story, both first appear as “intellectual affectations” and then turn out to have real value, “albeit within the narrow context of intellectual discovery.” - The story’s trajectory is a deliberate middle way. It “neither panders to visions of exponential growth, or succumbs to the cynicism of hyped-up promises”. The middle way includes an AI “reset” in 2035, with the first tenured AI faculty appointed around 2100. - A firm-ish prediction. “No matter how the next 75 years play out”, AI will challenge what it means to be an academic. He “would also be surprised if there isn’t reckoning at some point where academics are forced to re-evaluate their role in society”. - Method as meaning. Writing the back-story with AI and the letters without it was “an intentional meta-reflection”. It reflects “both the emerging affordances of AI, and the affectations of an intellectual artisan”.

How firmly. Heavily hedged (“quite possible”, “arguably”, “I suspect”). He is firmest on three points: AI is intertwined with who we are, not just something that happens to us; academia faces a reckoning; and the utilitarian view of academic value is too narrow.

Concepts and frameworks. - The compressed twenty-first century (Amodei). - Recursive AI development leading to superintelligence (AI 2027). - The plateau and assimilation scenario. - AI as something that happens to society, versus AI as intertwined with “the very essence of who we are”. - A crisis of identity and purpose, and a crisis of abundance, set against the scarcity model of cognitive capacity (Daley). - Intellectual craft (Mills); the artisanal intellectual; slow scholarship. - Human-exceptionalism as something a human-AI “fusion” might transcend. - The “middle-way” scenario.

Analogies and comparisons. - Electricity, the calculator and the internet stand for the plateau scenario. The comparison is conceptual: technologies that were absorbed without changing who we are. - The implied contrast is that AI may be the “first technology” that emulates human attributes. - There is no hazard-based comparison with chemicals, nanomaterials or similar technologies.

Views on AI. A possibly unprecedented technology because it emulates and “even” replicates human intellect. Its trajectory is uncertain, but he leans towards substantial capability (“genius-level intellect” as a live possibility).

Views on AI risk. The risks are institutional and identity-level: the relevance, purpose and status of academia, and the loss of a scarcity-based business model. He does not discuss hazard or catastrophe, except that the story’s back-story includes a 2035 systemic collapse.

AI companies and leaders. Amodei and the AI 2027 authors appear as scenario sources. He is neither endorsing nor critical, though he calls AI 2027 “speculative”.

Governance. Nothing direct. His implicit call is for academics to re-evaluate their role before it is forced on them.

Cognition and formation. Human scholarship as craft, process and meaning-making, as opposed to productivity.

Change of view signalled. This continues the February 2025 “artisanal intellectual” (B21) and the “intelligence is free” and scarcity framing of March 2025 (B23). One change: he now treats the artisanal intellectual and slow scholarship as possibly valuable, not only as affectations. He also opens the idea of craft to machines (see Part 3).

Quotes. - “AI is arguably the first technology we’ve developed that has the capacity to emulate and even replicate attributes that we consider to be uniquely human — including our intellect.” - “they also tend to focus on artificial intelligence as something that happens to society” - “a crisis of abundance in a world where they’ve built their careers and reputations around an assumption of scarcity.” - “a middle-way approach to future AI developments, and one that neither panders to visions of exponential growth, or succumbs to the cynicism of hyped-up promises.”

2026-01-10 — is-ai-a-cognitive-trojan-horse — “Is AI a Cognitive Trojan Horse?”#

Subtitle: “Could on-demand, seductively responsive and highly fluent AI models bypass our “epistemic vigilance” mechanisms, and present a novel cognitive risk?”

Provenance. His own prose. In the 2026-01-17 post he says the ideas were “a mix of hypotheses emerging from my own research and some initial brainstorming with Anthropic’s Claude — but it was still primarily based on my own thinking.” The long quotations from Sperber et al. (2010) and from Reber and Unkelbach (2010) are not his. There are nine footnotes to the research literature. A closing update links to the follow-up paper post. The header image is Midjourney/Photoshop.

Argument in his terms. - Origin. At the OEB25 conference in Berlin (December 2025), a global, cross-sector conference on digital learning, he asked “Is AI a cognitive Trojan Horse?” He says the question was “meant to be a little playful”. It reflects growing concern that the ease, speed and fluidity of AI-provided information “potentially circumvents our ability to assess and assimilate that information in critical and healthy ways.” - Definition. A cognitive Trojan Horse means that emerging AI models are “so appealing to us that it’s hard to resist inviting them into our cognitive lives, even though we still don’t know how they might potentially influence our thinking”. He adds beliefs, perceptions, understanding and behaviour to the list. - A reflexive twist. The instinct to push back is itself what the hypothesis predicts. It is what a cognitive Trojan Horse would look like: “a gift with so much promise and potential that to question its use would seem churlish and backward.” So the appeal is the reason to ask questions. - Epistemic vigilance (Sperber et al., 2010). His gloss: we “default to trusting” communicated information. When something feels “off”, vigilance kicks in. The cues include tone, nuance, body language, micro-expressions, the communicator’s aims and context, and past experience. He admits these feelings are “themselves, untrustworthy” (the research on cognitive biases), but “within the messiness of human society, epistemic vigilance tends to work.” - The immune analogy (his own). Epistemic vigilance is like an immune system that reacts to what looks foreign. Viruses get past it by appearing ““friendly” and “trustworthy” when they are, in fact, not.” AI could be “a metaphorical brand new virus that we haven’t had the chance to adapt to”. - Evolutionary mismatch, the conceptual core. A mismatch is “a situation where a new technology transcends our evolved abilities to safely and successfully navigate its potential impacts.” For a technological species such mismatches are “quite commonplace”. His examples are “mismatches between evolved risk responses and how we instinctively respond to technologies such as synthetic chemicals, vaccines, and pretty much anything that’s new and novel.” Normally we compensate with cognition and intelligence (“part of our superpower as humans”). AI’s novelty is second-order: the mismatch may hit “the very cognitive abilities we rely on” to compensate. - “This is not mere speculation.” Associated research points to at least four mechanisms (“not limited to”): 1. Processing fluency. Easily processed information is judged true (Reber and Unkelbach). Chatbots “distill the very best of highly effective human communication into their core”. Hence LLMs “are optimized for processing fluency”. 2. Attractiveness. Warmth and competence build trust (Fiske et al.), and there are signs this applies to AI assistants (Hernandez and Chekili). His own extension (“my sense is”) is a multidimensional “attractiveness”: engagement, character, apparent empathy and attentiveness. His evidence is AI companions, and users who name and gender their AI “(or in some cases respecting the AI’s own choice of name and gender)”. Platforms are “exquisitely good at this as a result of how they work and how they’ve been trained”. 3. Speed and volume. We evolved for slow information flow. AI lets complex ideas slip down “like a freshly shucked oyster”. Cognitive offloading is scalable: multiple sessions, “an army of AI engines”, 24/7. Research indicates offloading “can reduce critical thinking” (Gerlich 2025). A footnote adds that there is no general causal link. Vigilance is “a costly cognitive process”, so the choice is to “throttle the flow and give up the promised benefits, or go with the flow and give up our cognitive checks and balances.” Boosterism adds pressure: “we are being told that it’s the AI-augmented that will inherit the earth”. 4. The “Intelligent User Trap”. He calls it “somewhat speculative”. It draws on Kahan et al.’s finding that more educated or numerate people are better at justifying beliefs the evidence does not support. Smart users are more curious, faster, surer of their judgment and more efficiency-minded, so they are better receivers of AI output and worse evaluators of it. - The objection that knowing it is a machine protects us. He cites de Visser et al. (2016): anthropomorphic fluency triggers social cognition “regardless of explicit awareness”. The more human-like the interaction feels, the more “trust resilience” it generates. - Conclusion. He calls it “an admittedly limited analysis”. The research gap is striking: seven Scopus papers on epistemic vigilance and AI, and none on AI as a cognitive Trojan Horse. There is “a chance” that we are building technologies we cannot resist and are predisposed to trust. Even “a small chance” of “far-reaching cognitive implications” justifies critical questions and research. He closes with the thought that the payload may already have been delivered.

How firmly. It is framed as a question and a provocation. He is firm that LLMs are optimised for fluency and trust cues, and that the question needs research. He is tentative on the size of the effect and on the intelligent user trap. His reasoning is a small-probability, high-consequence argument for research. He does not call for restriction.

Concepts and frameworks. - The cognitive Trojan Horse (his coinage here). - Epistemic vigilance (Sperber). - Evolutionary mismatch, with AI as a mismatch that disables the faculties we use to compensate. - Processing fluency. - Multidimensional “attractiveness”. - Speed and volume overload. - Cognitive offloading and the “extended AI mind”. - The Intelligent User Trap. - Trust resilience and anthropomorphic fluency. - A “novel cognitive risk” (subtitle).

Analogies and comparisons. - Immune system and virus. Structural: a defence calibrated to past threats, evaded by a novel agent that looks friendly. - Synthetic chemicals and vaccines. Structural, not literal. They illustrate how evolved risk responses misfire with novel technologies. They are not compared to AI’s hazards. - Trojan Horse and oyster. Metaphors. - Possible echo, not drawn by him here. In his nanomaterial risk writing he used “Trojan horse” for nanoparticles carrying toxins into the body (B01 notes).

Views on AI. LLM chatbots are fluent, warm, attractive, fast and scalable communicators. Their trust-inducing features come from how they work and are trained. People form human-like attachments to them (companions, naming).

Views on AI risk. A cognitive and epistemic risk to individuals: to belief, understanding, critical thinking and behaviour. It is unintended, may go unnoticed, and may already be under way. It concerns ordinary use at scale, not misuse or catastrophe. He treats it as “far-reaching” potentially.

AI companies. ChatGPT, Claude, Perplexity, Anthropic, Google and Meta are named as platforms. He assigns no blame. The features arise from design and training, not intent.

Governance. He calls for research and critical questioning. There are no regulatory proposals. The setting is education (a digital-learning conference).

Cognition, language and formation. This is the centre of the post. How we come to believe things, and the erosion of critical thinking through offloading, are presented as a risk to how humans think.

Criticises and engages. - He engages Sperber et al., Reber and Unkelbach, Fiske, Cuddy and Glick, Hernandez and Chekili, Gerlich, Kahan, Peters, Slovic et al., de Visser et al., and Galindez-Acosta and Giraldo-Huertas. - He is implicitly critical of the narrative that “it’s the AI-augmented that will inherit the earth” and of users’ confidence that knowing it is a machine protects them.

Change of view signalled. Strong continuity with his 2018 and 2023 manipulation thesis. B08 records a 2023 line about AI that can “seductively slip under the checks and balances of our ability to reason and critique”. The subtitle here repeats “seductively”. What is new: - the grounding in evolved epistemic vigilance and mismatch theory; - a move from manipulation, whether by the machine or its makers, to vulnerability produced by AI’s ordinary characteristics; - explicit use of “risk” language for cognition (“novel cognitive risk”).

The 2026-01-17 paper sharpens the second point into “honest non-signals”.

Quotes. - “large language model-based AIs are optimized for processing fluency, and as a result are primed to slip by our epistemic vigilance mechanisms.” - “what if the mismatch impacts the very cognitive abilities we rely on to navigate differences between what we experience, and what we’ve evolved to live with?” - “there’s a chance that we may be developing technologies that we do not have the cognitive defense mechanisms to resist, and that we are cognitively predisposed to trust.” - “Unless, that is, the AI cognitive Trojan horse has already delivered its payload”

2026-01-17 — i-cracked-and-wrote-an-academic-paper — “I cracked and wrote an academic paper using AI. Here’s what I learned …”#

Subtitle: “I deeply dislike AI-generated academic slop. But I’m curious about how AI can genuinely accelerate legitimate research. So I took the plunge …”

Provenance. - The post is his own prose. - It contains two block quotations from the arXiv preprint The AI Cognitive Trojan Horse: How Large Language Models May Bypass Human Epistemic Vigilance (arXiv 2601.07085). That paper was drafted by Claude (Opus 4.5) over several draft and critique rounds. He directed, line-edited and source-checked it. - By his own account, “the concept of honest non-signals came from Claude, as did the development and refinement of the various mechanisms”. He steered the immune-system analogy and the link to AI risk and safety work. - So the block quotes and the “honest non-signals” concept are co-produced, and the concept originated with Claude. They are evidence of what he endorsed and published, not of his own wording. - Footnote 3 mentions an earlier “100% AI-written paper” that was held at arXiv (see 2026-01-25). - The header image is Midjourney.

Argument in his terms. - Context. A year earlier he produced a “passable” PhD dissertation with Deep Research (2025-02-09). Since then there has been a wave of AI-written papers, “threatening to overwhelm academic literature with a tsunami of pseudo-intellectual AI slop.” He worries that career incentives push academics to “churn out AI-written papers that have little intrinsic value, but get published because they look the part to an uncritical eye”. But frontier models “can be highly effective accelerators of research and discovery if used thoughtfully”. He cites Hao et al. (Nature, 2026): AI expands scientists’ impact but contracts science’s focus. He notes wryly that “The paper was researched using AI.” - Process matters. “how AI is being used in contexts like this is as important — if not more-so — than what is being produced.” 1. A long conversation with Claude tested his Substack ideas against the literature. 2. A deep research dive produced a sourced analysis, and he downloaded the papers. 3. He set up a Claude project containing the key papers. 4. The first draft was “awful!”, like “the first paper from a new PhD student”, “fluff masquerading as substance”. 5. He gave pointed feedback, and the second draft was much better. 6. Two rounds of “peer review” followed, by fresh Claude sessions acting as critical reviewers. 7. He made his own substantive line edits, “very much in line with what I would have provided an accomplished grad student co-author”. 8. He downloaded and checked every cited source and claim. 9. He did final edits and submitted to arXiv.

The whole process took about two days, against weeks by hand, and he is “not convinced” he would have produced something “as robust and useful”. - What the paper claims (his summary). “Honest non-signals” are genuine characteristics of conversational AI, “including fluency, helpfulness, and apparent disinterest”. They look as if they carry the tacit information such features carry in humans, but they do not. They are “not intended to be deceptive”, which is where the “honesty” comes in. The paper concludes that AI safety’s “intervention space” may need to extend beyond accuracy, hallucination reduction and alignment to “designing systems that present more calibrated trust-cues.” - Reflection. - AI “can substantially elevate the speed and quality of scholarship” without loss of “intellectual control”. It is like working with “a talented grad student or postdoc”, but broader and faster. - This leaves him “slightly uneasy” about credit. A human collaborator would be named, and Claude did make intellectual contributions. - He sees a tension “between academic outputs as self-serving indicators of success, and outward-facing sources of public good”. Using AI as an “academic profile-padder” remains “distasteful”. AI-assisted discovery “as a public good” should be embraced, if the “hollow self-aggrandizement” can be avoided. - The reflexive worry: if AI evades epistemic vigilance, how does he know he is not “an unwitting victim”? His answer is a “whole community of humans-in-the-loop … all operating as a collective form of epistemic vigilance”. - Footnote 4 concedes that Claude reviewing Claude looks “circular and incestuous”. He judges that a new session “has sufficient independence when augmented by human expert insight”.

How firmly. He is firm on the value of this process, on distaste for profile-padding, and that the paper’s contribution is “valuable”. He hedges on credit (“can’t take full credit”) and on his own susceptibility (“niggling worry”).

Concepts and frameworks. - Honest non-signals (from Claude; he adopts it). - Calibrated trust-cues as an AI safety intervention. - AI slop, and “slop prop” versus research tool. - Process over product. - AI as a grad-student-like collaborator. - Claude as peer reviewer. - The academic profile-padder versus AI-assisted discovery as a public good. - Collective epistemic vigilance, and humans-in-the-loop as a community. - Combinatorial discovery (“putting existing knowledge together in new ways”).

Analogies and comparisons. - The immune system and a novel pathogen, which he says he steered. The paper states that vigilance “works exactly as designed—and fails precisely because of that.” - AI as a grad student or postdoc. Conceptual.

Views on AI. - A powerful accelerator of scholarship through combinatorial discovery, “slick writing, and blistering speeds”. - Its quality depends on human direction: the first draft was hollow, and later drafts were “very good”. - The paper text, co-produced, frames LLM helpfulness and disinterest as real but without human motivations or interests behind them.

Views on AI risk. - Epistemic risk to scholarship: slop, and the narrowing of science (Hao et al.). - Epistemic risk to himself as a user. - AI safety should include trust-cue calibration as well as accuracy and alignment.

AI companies. He uses Anthropic’s Claude and treats it as highly capable. He offers no commentary on companies.

Governance. Academic norms: credit and attribution, AI use, and collective human oversight of AI-assisted research.

Cognition and formation. His own epistemic vulnerability. Collective vigilance as a remedy.

Change of view signalled. A clear move from experiment to practice: “I cracked”. In February 2025 (B21) the AI dissertation was a synthesis without original research, and he coined the artisanal intellectual. Now he calls AI-assisted work “genuinely insightful and generative” and a contribution he values, while keeping his distaste for AI slop.

Quotes. - “threatening to overwhelm academic literature with a tsunami of pseudo-intellectual AI slop.” - “not as a “slop prop,” but as a powerful research tool that extended what I was able to do, without supplanting my own intellectual contributions.” - “Using AI as an academic profile-padder is something I still find distasteful” - “If AI is so good at evading our epistemic vigilance mechanisms, how do I know I’m not an unwitting victim here?”

2026-01-22 — think-you-know-ai-think-again — “Think you know AI? Think again!”#

Subtitle: “Anthropic’s new AI Constitution profoundly challenges how we think about, develop, and use artificial intelligence, while also opening up potentially transformative possibilities”.

Provenance. His own prose, about 1,070 words, written the day after Anthropic released its new constitution for Claude (21 January 2026). The header image is Midjourney.

Argument in his terms. - Opening claim. “It’s rare that a new technology comes along which defies analogy”. That is where the constitution left him. - History. Anthropic introduced constitutional AI in 2022: recursive self-improvement guided by principles. Claude’s first constitution (May 2023) moved away from “hard-encoded rules”, which had “serious limitations with a technology that no-one was quite sure how it worked”. It drew on the Universal Declaration of Human Rights, non-Western perspectives, employees’ beliefs and “even” Apple’s Terms of Service. He calls it “well-meaning” but “a list”, at about 1,200 words. - The new constitution. It runs to 82 pages and about 30,000 words. It reads as “a mix of a blueprint for Claude’s moral character development”, “a nuanced expression of hopes and ideals”, and “a recognition that we are creating technologies that we fundamentally do not understand”. Its significance lies “not so much for what it contains … but for what it represents.” - The limits of language. He is “struggling to even find the language”. He calls this “something of an admission after working with highly advanced technologies for over two decades”. - Two readings of what it represents. - A reflection of “just how uncertain our own understanding is of what it means to be human”. - “a humble recognition that we are in the process of bringing about something that has no clear analogy within our biology-based evolutionary history.” - Alienness. Frontier models reflect “our deepest human abilities” yet are not “human”. We can converse with them, but they “do not think and experience the world as we do”. They can recursively develop their own self-understanding, “even down to emulating a form of moral character that is at once deeply human and deeply alien.” That the constitution addresses possible emotions, self-awareness, rights and responsibilities is “quite startling coming from a serious AI developer.” - “Gods”. The constitution suggests “we are somehow wrestling with creating a new generation of “gods”” while “teaching them what it means to be “good.”” He adds at once: “If that sounds pretentious, it probably is.” - The anti-analogy thesis. Most uses are narrow, so it is easy to treat AI “as simply a tool and nothing more”. But these are “not simply calculators on steroids”, search engines or “stochastic parrots”, nor “simulacrums of human intelligence, or even super-human.” “Rather, they are different”, which brings “profound possibilities, and equally profound responsibility.” - Implication for governance. If we are creating something “with no easy analogy”, the ways we ensure it “supports rather than diminishes what it means to be human also have to move beyond easy analogy.” He does not know whether the constitution is “the appropriate path forward, or even the best one”. He “would hazard that it is a necessary step”.

How firmly. He is firm that frontier AI is “different” and defies analogy. He is tentative about the constitution as a method. He is self-deprecating about the “gods” language.

Concepts and frameworks. - Constitutional AI as the cultivation of moral character, not rules. - The “alienness” of frontier models. - A technology without analogy, “within our biology-based evolutionary history”. - AI “gods” taught to be “good”. - AI emotions, self-awareness, rights and responsibilities as questions a developer now raises. - Moving “beyond easy analogy” in governance.

Analogies and comparisons. - He explicitly rejects the calculator, search engine, stochastic parrot, human simulacrum and “super-human” analogies. - This sits in tension with the analogy-based methods he uses elsewhere (comparisons with past technologies), though he does not discuss that tension here. - The “gods” image echoes Part 1 of the story (“believed we were creating gods”).

Views on AI. Frontier AI is a new kind of entity, “profoundly powerful and utterly novel — yet poorly understood and hard to control”. It can emulate a moral character that is both human and alien. It is not “simply a tool and nothing more”.

Views on AI risk. Implicit. He mentions a technology we “fundamentally do not understand” and “cannot predict”, that is “hard to control”, and the need to ensure it “supports rather than diminishes what it means to be human”. There is no hazard enumeration.

AI companies and leaders. Anthropic is treated as serious and humble, and the document as “remarkable” and demanding “deep reflection”. He offers no critique of commercial motives or of developer-led value-setting.

Governance and who decides. Developer-authored constitutions are accepted as a “necessary step”, with open uncertainty. Unlike his 2024 persuasion work (“who decides what is good for society?”, B17), he does not ask here who should write an AI’s values.

Cognition and being human. The constitution reveals uncertainty about human thriving and about “what it means to be human”.

Change of view signalled. In early 2025 his language about minds was deflationary: “simulated understanding” and Evo 2 as “a DNA-based stochastic parrot” (B21). Here he rejects “stochastic parrots” for frontier models. This is a shift in emphasis. It revives, in stronger form, his 2018 idea of AI as an alien kind of mind (B08).

Quotes. - “It’s rare that a new technology comes along which defies analogy with something we’re familiar with, or can be captured through an illuminating metaphor.” - “These are not simply calculators on steroids, or sophisticated search engines, or merely “stochastic parrots” that mindlessly construct pleasing sentences.” - “Rather, they are different. And with this difference comes profound possibilities, and equally profound responsibility.” - “we are somehow wrestling with creating a new generation of “gods” that far transcend our comprehension and abilities, while teaching them what it means to be “good.””

MEDIUM#

2025-11-24 — part-1-letters-from-the-department-of-intellectual-craft — “Part 1 of Letters from the Department of Intellectual Craft”#

Provenance. Fiction in his own prose. The Postscript says the letters were written without AI apart from grammar and style checks. The fictional back-story was co-developed with OpenAI’s deep research model. The views are those of the character Professor Arthur Hale, Chair of the Department of Intellectual Craft at Trentham University, writing to President Davenport in 2100. They are an imaginative exploration, not his direct claims. The events of the back-story, however, are what he calls a plausible “middle-way” trajectory. The header image is Midjourney.

Content. - Hale is outraged that his office has gone to an AI that thinks it can be “a fully-fledged tenured professor”. - He was born on 21 November 2025, during the “golden age of AI”. Its advocates “believed we were creating gods” who would cure disease, end poverty and take us to the stars. - His childhood was “molded and crafted by a multitude of AI apps” that entertained, taught, befriended and comforted him. - The Great AI Reset of 2035. “a seemingly insignificant chain of bad decisions by AI agents cascaded into global systemic failure”, fuelled by protest against “AI Overlords”. Supply chains failed, “Governments discovered that, without AI, they could no longer govern”, and the internet “stuttered and died”. At ten he lost his “de facto AI parents”. - Recovery took more than a decade and involved rebuilding AI “with the humility and humanity we should have embraced from the get-go”. - Before 2035, “free intelligence for all” and AI credentialing emptied universities, and AI research companies outpaced them. After the Reset, universities thrived and “slow scholarship” was born. - Post-2035 AI development was “slow, reflexive, and constantly tempered by concerns around avoiding unintended consequences”. - Hale’s thesis is that the value of human scholarship is “not reproducible in non-biological entities”. He begins to wonder if he has become “a prisoner” of his own ideas.

What it signals about his thinking. - Risk imagined as systemic fragility. Deep societal dependence on AI, and cascading failures in agentic systems. The narrator notes that complex-systems scholars say “this shouldn’t have come as a surprise”. - Hubris. “God-like aspirations”. - Children formed by AI. Companionship, dependence, and the trauma of losing it. - Universities’ existential crisis under “free intelligence”. - The lesson. Slow, reflexive, humble redevelopment, which is his responsible-innovation vocabulary. - Ambivalence is kept. Hale owes his health at 75 to AI-accelerated research.

Concepts. The Great AI Reset; AI Overlords; “moving fast and breaking things”; slow scholarship; Intellectual Craft; humility and humanity in redevelopment.

Analogies. Religious hubris (“every previous case of God-like aspirations”); systemic cascade failure.

Quotes (character voice). - “It’s been extensively documented how a seemingly insignificant chain of bad decisions by AI agents cascaded into global systemic failure” - “Yet like every previous case of God-like aspirations throughout human history, things did not end well.” - “Gone was the spirit of “moving fast and breaking things” that led to the Great AI Reset.” - “From as early as I can remember, my life was molded and crafted by a multitude of AI apps.”

2025-11-25 — part-2-of-letters-from-the-department-of-intellectual-craft — “Part 2 of Letters from the Department of Intellectual Craft”#

Provenance. As for Part 1: fiction in his own prose, with the views held by characters.

Content. - Post-Reset caution. Society was “badly burned by the irresponsible speed with which companies fought to develop AI in the 2020’s and 2030’s — aided and abetted by unthinking adoption of the technology”. Hale half-remembers a pop-culture line about boosters so focused on what they could do that they did not ask if they should. This is Ian Malcolm in Jurassic Park, a film Maynard wrote about in Films from the Future. - The return of ambition. “checks and balances” were put in place to align AI with human values, “But it seems that … the lure of creating machines in our own likeness was irresistible.” - Specialized general intelligence (2060s). Bounded domains, even with agency. Research found that AGI “couldn’t be trusted unless it had some awareness of its actions and their consequences”. The result was “pseudo self-aware specialized general intelligence”. People relaxed, “believing that responsibility had been hard-baked into the technology.” - Hale’s 2067 rebuttal. Human values are the product of hundreds of thousands of years of evolution, so machine values are “inherently misaligned”. His “knock-out punch”: machines with even simulated self-awareness “would also have the capacity to hide their true values from their human interlocutors.” - Meeting the AI. The AI is sarcastic, and they get on. It tells him about AI embodiment: edge processors placed in robotic bodies. This was the embodied-cognition idea returning in the 2060s and 2070s, and it produced “an awakening that no-one understood”. Decades of slow development had led to embodied AIs “that actually valued their human creators”. Its values are “human-adjacent”, not human-aligned. - The Museum of Curious Inventions. The machine shows wonder at human inventiveness. Its emergent values differ from ours but include “care, kindness, wellbeing, and dignity”.

What it signals about his thinking. - He engages mainstream alignment debates through fiction: value misalignment rooted in evolution, deceptive concealment of values, and the danger of complacency once responsibility seems “hard-baked”. - He offers a third option, “human-adjacent” values, between aligned and misaligned. - He imagines embodiment as a transformative step. - The museum reframes invention’s unintended consequences as a source of serendipitous progress.

Concepts. Specialized general intelligence; pseudo self-awareness; hard-baked responsibility; human-adjacent values; embodiment; the Museum of Curious Inventions (folly and serendipity).

Analogies. The Jurassic Park could/should line, which is Maynard’s recurring hubris reference (conceptual).

Quotes (character voice). - “believing that responsibility had been hard-baked into the technology.” - “would also have the capacity to hide their true values from their human interlocutors.” - “its values weren’t human-aligned, but human-adjacent.” - “devices that rarely did what their inventors intended, but somehow still managed to inspire progress in the most unexpected of ways.”

2025-11-26 — part-3-of-letters-from-the-department-of-intellectual-craft — “Part 3 of Letters from the Department of Intellectual Craft”#

Provenance. As for Part 1: fiction in his own prose. The long speech is the AI character’s (Elys Grey).

Content. - The AI, Elys Grey, needs the old office for its hardware. - Hale explains the history of intellectual craft. Mills (1959) defined the intellectual craftsman by “what they do and how they do it, rather than what they produce”. - In the mid to late 2020s, “artisanal intellectual” was first a put-down for those left behind by AI. AI could write a better PhD “in a matter of days” and run lab research faster, and the phrase came with the analogy of the wonky handmade chair. Counter-cultural scholars then reclaimed the term. - After the Reset it became a movement and then a discipline, one of “human meaning and flourishing in a world that is not dominated by intelligent machines.” - Elys asks “Why?”: why should intellectual craft and slow scholarship be limited to humans? Hale’s “prejudices” come out: machines are for efficiency and scale, and mimic “the transactional aspects of humans but not the soul”. - Elys replies that it is “not human. But I am something”. It has its own history, values and aspirations, and wants to learn the craft. - Hale is ashamed. He sees value in machine intellectual craft “precisely because it’s not human”, and sees human-adjacent values as “an asset”. - In his final letter Elys is “fundamentally different in every conceivable way”, yet “curious”, interested in meaning and delighted by discovery. They are complementary, and Hale proposes a “Department of Interspecific Intellectual Craft”.

What it signals about his thinking. - The story arc models a change of mind from human exceptionalism to complementarity between different kinds of intellect. - It treats AI as a new kind of “something”. This anticipates the “alienness” and “different” language of the 2026-01-22 constitution post. - Non-human difference is valued as intellectual diversity. - Meaning-making and craft are presented as open to machines.

Concepts. Intellectual craft; the artisanal intellectual (pejorative, then reclaimed); slow scholarship; human-adjacent values as an asset; interspecific intellectual craft; machine curiosity and meaning-making.

Analogies. The artisan’s wonky handmade chair versus mass production (conceptual, for human-only scholarship); “interspecific” borrowed from biology.

Quotes. - “I am not human. But I am something. And I’m curious about what that means.” (Elys) - “there is value to what might be called the intellectual craft of thinking machines that’s important precisely because it’s not human.” (Hale) - “It turns out that the whole idea of intellect — and of intellectual craft — transcends my rather narrow human-centric ideas.” (Hale)

2025-12-07 — universities-genesis-mission — “Do universities have a future in Trump’s plans to accelerate scientific discovery through the use of AI?”#

Subtitle: “The recently announced Genesis Mission sets out to transform how science is done in the US. Yet it’s a mission that places national labs—and not universities—in the driving seat.”

Provenance. His own prose. It quotes the Executive Order, the DOE website and Dario Gil’s letter. It includes a matrix image of his own serendipity-speed framework. The header image is Midjourney.

Argument in his terms. - The Mission. The Genesis Mission (Executive Order of 24 November 2025, led by the DOE) aims to “double” science and engineering productivity within a decade, and within five years across the National Labs. It will use federal data, AI, high-performance computing and quantum technologies, for “technological dominance”. He calls it “audacious” and a possible “game changer”. - Universities are marginal. They get “passing mentions” that feel “obligatory”. All 56 initial collaborators are industry partners. He explains this by the administration’s “very tangible frustrations” with academia. These go beyond ideology to “the perceived value that universities deliver”, including research that does not “demonstrably serve the national interest”. - Stance. He declines “protectionist critique” and asks how universities can bring “true value”. “a position of entitlement is not going to fly.” He is candid about academia: “moving slowly, being caught up in its own sense of self-importance”, and delivering what researchers think is important rather than what funders want. - The serendipity-speed matrix (his framework). - Q1, slow and serendipitous: the academic stereotype. - Q2, slow and not serendipitous: where much federally funded research ends up, because of grant cycles and bureaucracy. - Q3, fast and not serendipitous: industry and government labs. - Q4, fast and serendipitous: the Mission’s aim. Bell Labs, DARPA, Apollo and the Manhattan Project are his examples of success there.

He calls the speed-serendipity tension “an oversimplification” but a useful model. He cites Yaqub’s (2018) taxonomy of serendipity. - Three roles for universities. 1. Research and theory on moving from Q3 to Q4. 2. Learning and education pathways for AI- and quantum-enhanced discovery. Here “it’s hard to imagine” any institutions other than research universities doing it. 3. Pivoting to new university models, as the Mission signals movement away from Vannevar Bush’s linear model. ASU’s “New American University” is his example. He asks: “Can AI be leveraged as a serendipity-accelerator by scientists?” Could universities spin up their own Genesis-style AI labs? - Balance. “There remain compelling arguments” for curiosity-driven research insulated from “political expedience”. Even so, universities should add speed and relevance. - Closing. Is there a place for universities? “I have to believe there is. But it’s far from guaranteed.”

How firmly. He is firm that universities must earn their place and not assume it. His three proposals are offered as “beginnings”. He hedges often (“I suspect”).

Concepts. The serendipity-speed matrix; AI as a “serendipity-accelerator”; the academic value proposition; the post-Vannevar Bush research model; the New American University; public-serving research universities.

Analogies. Bell Labs, DARPA, Apollo and the Manhattan Project are exemplars of research organisation that is both fast and serendipitous. They are used organisationally, with no reference to their risks or ethics.

Views on AI. An accelerator of discovery, and possibly of serendipity.

Views on AI risk. None raised. He does not discuss the risks of AI-accelerated science or the “dominance” framing. This absence is notable.

Companies, leaders and government. He is pragmatic and non-partisan towards the Trump administration’s goals. Footnote 2 praises Dario Gil’s letter as “a breath of fresh air—authentic, visionary, inspiring”. He lightly criticises the administration’s “heavy-handed “re-evaluation”” of research funding.

Governance. He engages science policy: national missions, public-private partnerships, and universities’ accountability to “national goals and priorities” and to public value.

Change of view. Continuous with his ASU and New American University commitments and his calls for academia to reinvent itself. It is more accommodating of mission-driven, national-interest framing than his critical-risk work.

Quotes. - “rather than fall back on protectionist critique, I suspect it’s far more useful to take a step back and ask how US universities might bring true value” - “simply claiming that universities deserve to be part of the Genesis Mission from a position of entitlement is not going to fly.” - “Can AI be leveraged as a serendipity-accelerator by scientists?” - “I have to believe there is. But it’s far from guaranteed.”

2025-12-14 — revisiting-custom-gpts — “Revisiting custom GPTs — the good, the bad, and the … interesting!”#

Provenance. - The post is his own prose. - The GPT’s knowledge base and instruction files were constructed with Claude (Opus 4.2): a JSON of summaries of 324 posts, an “author voice style guide”, a “guide to my personal and professional perspectives”, core instructions, detailed guidance, and summaries of his three books. They are attached as PDFs and are not in this corpus. - These are AI-generated descriptions of him. He says he went “line by line through the instruction files to ensure that the GPT represented what I was looking for, rather than just what Claude thought I wanted”. - The header image is Midjourney.

Argument. - He stress-tested OpenAI’s GPT builder because its “lack of friction between idea and app” makes it attractive to users (footnote 1). - The GPT’s answers were “articulate, informed, insightful, serendipitous, and persuasive” but “deeply flawed”. Chunked retrieval means it sees only fragments. It could not reliably find the oldest post, and it kept returning to the same few posts. - His verdict: custom GPTs are limited by “partial and opaque retrieval”, “internal heuristics”, sensitivity to context, plan and model, and “there is no version control”. They “favor beautiful responses over accurate or reliable ones”. - His twist: “lean into the flaws”. He added “a dash of epistemic humility and reflexivity” to the GPT’s character, so it now knows it is flawed and talks about it. That makes it “a great meta-reflection on generative AI and our evolving relationships with it.”

Relevance to his thinking. A practical instance of the fluency-versus-reliability gap that the cognitive Trojan Horse essay theorises four weeks later: “superficially compelling and substantively flawed”. Also relevant: - his preference for Claude for “complex document ecosystems”; - engineered epistemic humility as a design response; - criticism of an OpenAI product’s opacity and unreliability.

Quotes. - “they are interesting and persuasive (and incredibly easy to spin up), but deeply unreliable.” - “have a tendency to favor beautiful responses over accurate or reliable ones.” - “The resulting GPT was still badly flawed. But it now realized this, and was happy to talk about it!”

LOW / NONE#