B20 digest: 2024-11-06 to 2025-01-12#

What this batch is#

Ten weeks from Trump’s election to mid-January 2025, covering OpenAI’s “12 days” releases, o3, and Altman’s prediction that AI agents would “join the workforce”. Five of the eighteen posts are Modem Futura podcast posts. At the user’s request they are set aside, apart from a few paragraphs of Maynard’s own writing on governance in 2024-11-06. Three more posts are mostly other people’s words: guest reflections (Native American communities), a review of Gordon and Seth (BCIs), and a FAQ drafted mostly by ChatGPT. The rest is his own prose.

Main ideas#

1. Risk innovation, restated and tested on a consortium (2024-12-17). This is the clearest statement of his risk framework in this period: - Risk is a “threat to value”, existing or aspirational. - Value, not values: values “are important, but the former is more effectively operationalized in policy and decision-making”. - Eighteen orphan risks: “risks that are often overlooked because they’re messy, subjective, and hard to deal with”. They sit alongside quantifiable risks to health, the environment and finances. - Internal versus reciprocal threats: some risks come directly from an organisation’s own actions; others arise when it threatens other people’s value and that comes back to threaten it. - Co-opted tech and bad actors.

What is new is the application: a framework built for startups is used with an NSF biotechnology consortium (ATP-Bio), where workshops found ethics, perception and government and regulation dominant. Responsibility is enlightened self-interest, not moralising: success “is intimately intertwined with how they impact (and threaten what of value to) others around them”. Oversight is “only part” of the landscape. The case is biotech, not AI, and orphan risks are one tool within the value-based approach.

2. AI’s quieter threat: loss of meaning, joy and agency. Three posts form a thread. - The soul of science (2024-11-10). Reacting to an MIT study, later disavowed, which he flagged in a May 2025 update, he argues that “the soul of science lies in the delight and wonder of exploring the unknown”. Societal good must be defined more broadly than “short term financial or material gain”, or science becomes “a utilitarian tool to support a utilitarian world heading for a utilitarian future”. - The amanuensis thought experiment (2024-11-24). A three-actor model in which a primary agent tasks an AI, which in turn directs humans. The result is “AI-directed and human-executed implementation”, which can scale into AI-to-AI networks in which agency drains from people. He doubts that AI truly generates new ideas yet. He worries that organisations will adopt this model at scale without understanding its consequences for “human creativity and innovation”. - Joy (2024-12-29). A footnote makes joy an explicit criterion: advances “have a nasty habit of sucking the joy out of what we do without us realizing it!”

The risk is not catastrophe but erosion of what gives human work and identity value.

3. What AI is: ambient, quasi-personal, and rising or plateauing. - 2024-12-13. Three s-curves: capability (LLMs possibly plateauing), utilisation (accelerating) and perception (lagging, with mental models “stuck” in 2022). He describes AI as “a smart, engaging, and supremely patient artificial person integrated into everything you do”. - After o3 (2025-01-05 and 2025-01-07). Models are “pushing far beyond” the stochastic-parrot critique. He gives Altman’s agent prediction “a reasonable chance” of being right, citing simulated reasoning and control of external systems. - 2025-01-12. He uses “we” for his work with ChatGPT, because machines are “fundamentally different from passive devices”. He calls AI “limited and flawed” but “a game changer”, an amplifier of existing expertise.

4. Who steers the advanced AI transition. - The Trump administration (2024-11-06). He expects a “permissionless” turn. Responsible AI will then rest on “a tapestry of soft governance mechanisms” run by developers and consumers. He reports this as the new reality rather than arguing against it. - Universities (2025-01-07). He sets out his institutional analysis: - AI companies are “trying hard” but lack “breadth of vision and understanding”; - democratic governments have the mandate but lack “imagination, vision, or agility”; - research universities are best placed but “mired in tradition, convention, and self preservation”.

He proposes integrated, “domain-agnostic” work on three foci: where we live, what we do, and who we are. He rejects bolt-on AI and X programmes and wants billions from philanthropy. - Elsewhere. Agile, technology-agnostic education policy (2024-12-13). “tech bros” who forget “the Dunning-Kruger effect when it comes to governance and policy” (2024-12-29). AI firms short of staff for “responsible development” (2024-12-20).

5. Plausibility, history and hype. - BCIs (2024-11-17). Imagination disciplined by feasibility, “grounded in plausibility rather than hyperbole”, a technique he calls his own. - Geoengineering (2024-12-01). Nanoparticle measurement traced to Aitken’s Victorian dust counter, and his 2015 claim that “the key to moving forward safely, is to look back at what’s already known”. - Hype (2024-12-29). The pencil-and-paper test, and a warning against efforts to “manufacture problems” that retroactively justify technologies.

6. Being human, cognition and justice. - BCIs. He extends Gordon and Seth: neural data as AI training data; “mental monoculture” as a route to “manipulation of “group think” by dominant producers, controlling governments, or even AI systems”; authenticity as a question for any technology. - Being human. Its bedrock is not “an immutable truth” (2024-12-29). - Justice. A capability gap between those who learn to use AI and those who don’t (2025-01-12), and representational bias (Sora; Tuba City).

Concepts appearing#

Risk innovation and its parts (threat to value, value versus values, orphan risks, internal versus reciprocal threats, co-opted tech, bad actors). The pacing problem. The soul of science. Human as AI amanuensis and the three-actor model. The three s-curves. Ambient, quasi-personal AI. AI literacy as ways of thinking, and persistent failure modes. Agile, technology-agnostic policy. The advanced AI transition and its three foci. Soft governance and permissionless innovation. Plausibility over hyperbole. Mental monoculture. The pencil-and-paper test. Joy as a value. Human–AI “we”.

New or changed#

• A visible short-run shift on capability. On 13 December he gives weight to an LLM plateau. By 7 January, after o3, he takes agentic and AGI-adjacent trajectories seriously enough to demand billion-dollar university initiatives.
• Risk innovation extended. It moves from startups to multi-stakeholder research consortia, with first empirical workshop data, and the value-versus-values distinction is stated explicitly.
• A new concept. The “human amanuensis” introduces an organisational, agency-shifting view of AI’s effect on work.
• A milder tone towards AI companies than in B15. He credits their effort and bias fixes, but criticises their narrowness and thin responsible-innovation staffing.
• Universities and governance. His B15 faith in public universities now comes with open doubt. He treats Trump-era soft governance as the working reality and looks to universities and philanthropy rather than regulators.

Most important posts#

1. 2024-12-17 navigating-the-challenges-and-opportunities-of-advanced-biopreservation-technologies. The fullest restatement of risk innovation and threat to value, now tested empirically.
2. 2024-11-24 artificial-intelligence-agency-human-amanuensis. His original model of agency shifting from people to AI.
3. 2025-01-07 universities-need-to-step-up-their-agi-game. Who should steer the AI transition, and the three foci.
4. 2024-12-13 are-educators-falling-behind-the-ai-curve. The three s-curves, the nature of current AI, and agile policy in education.
5. 2024-11-10 is-ai-poised-to-suck-the-soul-out-of-science. Joy, wonder and a broadened definition of societal good as criteria for judging AI.

Runner-up: 2024-11-17 (BCIs), for plausibility-grounded ethics.