B16 notes: 2024-06-25 to 2024-08-18 (15 posts)#

Reading notes on Andrew Maynard’s Substack posts in batch B16. Only his own prose is treated as evidence. Quotes are exact, including his original typos and curly punctuation. (The Dune post contains invisible soft hyphens in some words; no quote below crosses one.) This batch contains no Modem Futura podcast posts.

Batch context. Summer 2024. OpenAI’s GPT-4o launched in May (he wrote about “hyper-anthropomorphism” then). Google DeepMind’s Ethics of Advanced AI Assistants paper came out in April, and Sutskever’s Safe Superintelligence in June. In this batch:

• He is at the WEF “Summer Davos” in Dalian for the Top Ten Emerging Technologies launch.
• On 13 August he gives a keynote in Bali at the IEEE International Symposium on Consumer Technology, titled “A New Science of Navigating Advanced Technology Transitions”. Two posts come out of preparing it.
• Four posts are about Elon Musk’s ventures (Neuralink twice, the Ad Astra school, and Optimus inside the humanoid-robots post).
• One is a published law-journal review (Jurimetrics) on Dune and AI.
• One is a guest post, not his.

“Advanced technology transitions” is the organising frame across the batch.

Relevance summary#

HIGH#

2024-06-30 — seth-is-conscious-ai-possible — “Is Conscious AI Possible?”#

Provenance. His own prose, written as commentary on Anil Seth’s preprint “Conscious artificial intelligence and biological naturalism” (PsyArXiv, June 2024). The long quoted passages are Seth’s, not his: “skin in the game”, autopoiesis, “many more ways of being mush”, “unprecedented ethical catastrophe”, “techno-rapture”, “cognitively impenetrable illusions of consciousness”, and the closing Frankenstein paragraph. He chose to end on Seth’s paragraph (“only fitting”). The header image is MidJourney. His own contributions are the framing, the interpretive asides marked “in my interpretation”, the thermodynamics remark, his elaborations on suffering and manipulation, and his remarks on companies and governance.

Argument in his terms. - Advanced AI is conflated with conscious AI. He says that under the scepticism about “conscious” LLMs there is a strong conflation of the two. He goes further: “much of the current wave of AI acceleration being pushed by companies like OpenAI and Anthropic is underpinned by a belief that future human flourishing is depended on superintelligent machines that are likely to exhibit some form of conscious behavior”. This is a firm characterisation of the labs’ worldview, and he offers no evidence for it. - Why Seth matters. Seth’s case against computational functionalism and substrate neutrality, and for biological naturalism, “would place a rather large wrench” in plans to reach advanced AI by scaling digital compute. He adds his own step. Even if developers say superintelligence does not need consciousness, “the types of agency, reasoning, and problem solving capabilities inherent in aspirational AI are hard to imagine without it occurring in some form.” So Seth’s argument calls into question whether AGI and superintelligence are “feasible — or even advisable.” - His own reading of computation. Consciousness might still be computational, but “the very definition of computation needs to incorporate the whole system”, not a split between substrate and algorithm. He treats the hardware/software model as a projection of our own technology onto biology. - A long-held suspicion. “I’ve long suspected that there is a strong thermodynamics argument to be made against the inevitable rise of superintelligent (and conscious) self-replicating and self-improving machines.” Seth’s account of living systems holding themselves out of thermodynamic equilibrium “places flesh on these suspicions.” This fits his 2018 view (see B08) that Bostrom-style superintelligence is implausible on current substrates. - Real artificial consciousness would be an ethical catastrophe. On Seth’s warning that real artificial consciousness would be an “unprecedented ethical catastrophe” he writes “I firmly agree.” He adds that it could create forms of suffering we might not recognise, in entities classed as “objects” and “possessions” outside moral discourse. - Conscious-seeming AI is the near-certain problem. He glosses Seth’s “cognitively impenetrable illusions”: “we may rationally understand that an AI is not conscious, but be instinctively incapable of acting on this knowledge.” This leaves us open to manipulation. He links it explicitly to his Ex Machina chapter in Films from the Future: “It’s a concern that I’ve previously highlighted in the book Films from the Future while discussing the film Ex Machina. And it’s one that I think is even more relevant now.” - Benefit and risk come from the same property. Conscious-seeming AI may bring real benefits (productivity, learning, health, wellbeing), and those benefits may arise “because these systems act as if they are conscious and aware.” He then asks what the “societal and personal price” will be. - Governance gap. OpenAI, Anthropic and others are racing “to produce ever-more human-like AI — with very little governance overseeing the subtler potential social implications of emerging systems that we can’t help but respond to as if they are conscious”. Integration into people’s lives seems “inevitable”. What is needed is “a level of society-wide discussion and action around responsible AI development that we have yet to see”. - Humility against hubris. He praises Seth’s humility against the “hyperbolic and hubristic proclamations coming from proponents of advanced AI”. The paper is a conversation starter about not making serious mistakes because we think “when playing with the future, it’s OK to simply go fast and break things because we’re building something new.”

Concepts and frameworks. Seth’s terms, which he adopts: computational functionalism, substrate neutrality, biological naturalism, autopoiesis, real versus conscious-seeming AI. His own contributions: a whole-system definition of computation; a thermodynamic argument against self-improving superintelligence; hubris versus humility as a test of how responsible AI discourse is; “go fast and break things” as the attitude he rejects.

Analogies and comparisons. Frankenstein, through Seth: the sin is giving consciousness, not life. Ex Machina, through his own book. The hardware/software analogy is critiqued as anthropocentric. All are conceptual.

On AI. Current AI is not conscious, and real artificial consciousness is probably out of reach on digital substrates. Conscious-seeming AI is inevitable, “and one which some would argue is already here”. Human-like appearance is the new and consequential feature.

On AI risk. Two kinds: - Low-probability, high-stakes: a moral catastrophe if real consciousness arises “by design or accident” (Seth’s phrase). - High-probability: psychological exploitation and relational dependence on conscious-seeming systems, “by accident or by (corporate) design” (Seth’s phrase, which he amplifies).

He frames these as ethical and social risks, not existential ones.

On companies and leaders. OpenAI and Anthropic are named twice, as drivers of acceleration and of a “race” to human-like AI. He criticises proponents’ hubris and “technocentric visions of an AI future”.

On governance. Governance of the subtle social effects is almost absent. He calls for society-wide deliberation and action on responsible AI development. He offers no mechanism.

On cognition and formation. Instinctive response overrides rational knowledge. This cognitive vulnerability underlies manipulation.

Engages. Anil Seth (strong endorsement). AGI, singularity and superintelligence proponents (criticised).

Change of view. None. He reaffirms the 2018 manipulation thesis and his earlier doubts about superintelligence.

Quotes. - “I’ve long suspected that there is a strong thermodynamics argument to be made against the inevitable rise of superintelligent (and conscious) self-replicating and self-improving machines.” - “we may rationally understand that an AI is not conscious, but be instinctively incapable of acting on this knowledge.” - “with very little governance overseeing the subtler potential social implications of emerging systems that we can’t help but respond to as if they are conscious”

2024-07-13 — ai-choice-engines-sunstein — “AI Choice Engines, Paternalism, and Behavioral Manipulation”#

Provenance. His own prose. The following are quoted, not his: - Cass Sunstein’s commentary in Humanities & Social Sciences Communications, which he calls “Nature: Humanities & Social Science Communications”; - Thaler and Tucker’s 2013 HBR article (he spells it “Thayler”); - the Google DeepMind Ethics of Advanced AI Assistants paper; - in an italic update, a Mira Murati (OpenAI) interview quote he found via Mark Daley’s Substack.

The three quadrant diagrams are his own (“quickly sketched out”).

Argument in his terms. - He sets out Sunstein’s case. Information deficits and behavioural biases lead to poor welfare choices. Blanket nudges such as labels fail because people differ. AI Choice Engines could personalise corrections. This raises questions of paternalism and manipulation. He frames it as the “could” of AI Choice Engines against the “should” of technologies that, while designed to raise welfare, “risk threatening value in unexpected ways”. That is his risk-as-threat-to-value language. - He pushes further on manipulation, linking his republished Ex Machina chapter. Manipulation can be unintended: a feedback loop between a user and “Claude or ChatGPT” produces nudges that are not in the person’s interest. Or it can be intentional: images, messaging and engagement that exploit cognitive biases to benefit others. - Core claim (dual use): “the capabilities that make socially beneficial AI Choice Engines viable are the same as those that make AI-driven persuasion and manipulation possible.” - Quadrant framework. The axes are value to the individual and value to the “agent” that develops and deploys the engine (government agency, corporation, nonprofit): - Self-Determination: low value to both. Little incentive to build persuasive engines, so more latitude for the individual. - Empowerment: high value to the individual, low to the agent. The apparent ideal, but “the notion of an agent that minimizes value to itself in favor of others is highly naive, even in the case of governments.” - Manipulation: high value to the agent, low to the individual. - Transformation: high value to both. A win-win, but at risk of “brittle solutions” where the two kinds of value are in precarious tension. - Forcing functions. Movement out of Self-Determination is “extremely likely”, because “technological innovation and advances are inevitable given our human nature” together with societal and economic dynamics that are “near-impossible to resist”. The question is “how and when”, not whether. - The economic gradient. In an unconstrained environment, value to agents dominates, “simply because this is where the power and impetus lies”. This “leaves individuals as engines of value creation rather than the primary recipients of created value”. He cites data monetisation as a precedent that has already happened. “In other words, there’s likely to be an economic gradient that pulls the impact of future AI Choice Engines toward being used for manipulation rather than empowerment.” It need not be malicious: “This may not be intentional or even malicious.” - Governments are not exempt. His example is a federal agency nudging diets through AI. The agency’s own value includes power, influence, ideology-based policy and stakeholders. This is where paternalism arises: “who decides what is good for individuals”. “the irresistible pull toward power and influence is likely to move even governments toward the manipulation quadrant without appropriate checks and balances in place.” For-profit firms slide faster because of fiduciary duty. Nonprofits always have an organisational value proposition. - Remedy. Guardrails, “well-crafted policies and other barriers”, and “new thinking on how to resist the economic gradient toward manipulation” so as to push engines toward Empowerment or Transformation. The urgency grows as advanced AI assistants become choice engines, “including engines that may, one day, serve the goals of machines as well as humans.” - In the update, OpenAI’s CTO acknowledges major persuasion risks. His comment: “Sounds like the bottom right quadrant!”

Concepts and frameworks. - AI Choice Engines (from Thaler and Tucker, and Sunstein). - The individual-value versus agent-value quadrant (Self-Determination, Empowerment, Manipulation, Transformation). - Forcing functions. - The economic gradient toward manipulation. - Individuals as engines of value rather than recipients of it. - Paternalism and who decides. - Unintended versus intentional manipulation. - Risk as a threat to value. - Dual-use influence capability.

Analogies and comparisons. Data monetisation in the platform economy is a literal precedent for value being extracted from individuals. Nudges and labels as ‘mass’ interventions (Sunstein) are a literal policy comparison.

On AI. Advanced AI assistants are personalised engines of influence over belief and behaviour. The new feature is fine-grained, individually tailored influence at scale.

On AI risk. The emphasis is structural and political-economic, not about individual bad actors. Incentives and power gradients make manipulation the default, and paternalism by well-meaning institutions is a related risk. He offers a speculative extension to machine goals. He treats the risk as serious and increasingly urgent.

On companies and leaders. For-profit developers are structurally pulled toward manipulation. OpenAI’s own leadership acknowledges the risk (Murati).

On governance and who decides. Regulation and checks and balances are necessary, and apply to government deployers as well as firms. He is suspicious of any institution that claims to act purely for individuals. He endorses Sunstein’s call to scrutinise choice architecture for deception and manipulation.

On cognition and formation. Cognitive biases are both the target of correction and the surface of exploitation. Feedback loops between user and chatbot can shape behaviour without anyone intending it.

Engages. Sunstein (constructively; “it stops short”), Thaler and Tucker, the DeepMind authors, Murati.

Change of view. No reversal. It moves his 2018 individual-psychology account of manipulation (the Ex Machina chapter) toward an incentive-and-power account.

Quotes. - “the capabilities that make socially beneficial AI Choice Engines viable are the same as those that make AI-driven persuasion and manipulation possible.” - “there’s likely to be an economic gradient that pulls the impact of future AI Choice Engines toward being used for manipulation rather than empowerment.” - “the irresistible pull toward power and influence is likely to move even governments toward the manipulation quadrant without appropriate checks and balances in place.”

2024-07-21 — artificial-intelligence-dune-villeneuve — “Artificial intelligence is conspicuous by its absence in Denis Villeneuve’s Dune: Part Two. And this is important”#

Provenance. His own prose. A short 2024 introduction, then the full text and footnotes of his published review: Jurimetrics 64(2), Winter 2024, 163–167 (the ABA’s law, science and technology journal). It develops his “scrappy” Substack post of 2024-03-03. Quotations from Frank Herbert’s Dune and from critics in the footnotes are not his. The header image is Midjourney, which he chose deliberately to evoke “a future without AI on a desert planet”.

Argument in his terms. - Between Dune: Part One (2021) and Part Two (2024) “the world as we knew it changed”. ChatGPT turned AI from “side conversations amongst small groups of experts” into a matter of global attention. - Herbert’s universe has banished “thinking machines” (the Butlerian Jihad). He foregrounds the Reverend Mother’s line that people who turned their thinking over to machines were enslaved by “other men with machines”. The films omit it, but the backstory shapes them. What it means to be human is defined by “biological and psychological manipulation” (spice) rather than computer augmentation. - He is openly a minority critic of the films as entertainment, but reads them through “cinema as a mirror through which to better understand ourselves”. - On present AI. Present systems “are not in any way self-aware, or conscious, or able to “think” in any conventional sense.” But capabilities are approaching machines that “simulate human intelligence to the point that whether they are conscious or not may become moot.” - Main concern. AI “could disrupt society in complex and unpredictable ways as it allows increasingly higher levels of decision-making to shift from people to machines”. His examples are autonomous vehicles, legal analysis, medical decisions, “manipulating consumer behaviors” and critical infrastructure. “we are already irreversibly integrating AI into every aspect of our lives.” - Benefits are real and not trivial: medicine and scaled education. He notes “a seductiveness to AI platforms that have a mastery of the written word that far surpasses what most of us are capable of”. - The being-human question. “Are we selling our souls to the promise of a future of artificial intelligence convenience as thinking machines become commonplace?” His answer: “I don’t think we are. However AI changes us, I suspect that humanity is sufficiently adaptable and resilient to hold onto what makes us “us””. But he insists that the question of how AI could diminish who we are must not be ignored. - The films push us to ask who we are, what role thinking machines should play, and how they “might be governed” so that humanity thrives. They help move debate beyond “uninformed ideas and polarizing polemics”. - Neutral between options. Whether we decide benefits outweigh risks “or that we need to collectively slow the AI juggernaut”, films let us imagine alternatives and steer “toward what we want, rather than what we are resigned to accepting”. The present is “a profound advanced technology transition that is being driven by AI.” - Footnote 8 (his summary of Zachary Pirtle’s essay): Dune’s inhabitants are not anti-technology but “are opposed to technologies that lead to decision-making being relinquished to machines in ways that threaten what it means to be human.” Footnote 9 points to themes of “culture, belief, ideology, manipulation, and political power”.

Concepts and frameworks. Advanced technology transition (AI-driven). Cinema as a mirror. Decision-making relinquished to machines. What makes us “us”. Irreversible integration. Steering toward what we want rather than what we are resigned to.

Analogies and comparisons. The Butlerian Jihad as a fictional precedent for society-wide rejection of a technology (conceptual). Spice-based human enhancement versus computer augmentation (conceptual). No literal comparisons with other technologies.

On AI. Emulators, not thinkers, but ones whose simulation may make the question of consciousness moot. Their command of written language is a distinctive and seductive capacity.

On AI risk. Society-level disruption from displaced decision-making, which is complex, unpredictable and already irreversible. The risk of diminishing who we are is real but, in his judgement, survivable because humans are resilient.

On governance. He raises it as an open question and does not prescribe. He is explicitly even-handed between proceeding and slowing down. His emphasis is on public imagination and nuanced conversation.

On cognition and formation. He touches on the seductiveness of machine command of language and on outsourcing thinking. The Dune line about turning thinking over to machines is the frame he chooses.

Change of view. None substantive. This is the formal, published version of his March post.

Quotes. - “we are already irreversibly integrating AI into every aspect of our lives.” - “I don’t think we are. However AI changes us, I suspect that humanity is sufficiently adaptable and resilient to hold onto what makes us “us”” - “how we might steer it toward what we want, rather than what we are resigned to accepting”

2024-08-07 — are-humanoid-robots-really-the-future — “Are humanoid robots really the future?”#

Provenance. His own prose. Two passages from Figure CEO Brett Adcock’s “master plan” are quoted. He cites the “sci-fi feedback loop” concept from colleague Rizwan Virk.

Argument in his terms. - Signal of changed expectations: “If you’d asked me a few years back whether we’ll be living side by side with humanoid robots in the near future, I’d of laughed.” Companies are now betting on it (Figure 02, built with OpenAI, and Tesla’s Optimus), but “may still be living in a science fiction-fueled fantasy.” - The companies’ vision: replace manual labour, fill workforce shortfalls, raise output, take on mundane or dangerous tasks, go to other planets. Their design logic is “to build machines that understand and interact with a world made for humans and inhabited by them.” To sci-fi-raised engineers this feels like “manifest sci-fi destiny” (Virk’s “sci-fi feedback loop”). - Main claim. Technical hurdles are “fiendishly difficult, but not intractable”. “But the biggest barriers are likely to be societal rather than technological.” The key question is: “Will people tolerate mechanical human replacements?” - Jobs. Automation has historically brought protest and then acceptance. But at the scale imagined, and because “unlike previous waves of automation, these robots are more able than most workers to learn new skills”, he expects possible “a Luddite-like backlash against humanoid robots in the workplace that, unlike the original Luddite movement, actually succeeds.” - Safety as perception. How safe robots will be around people is unknown. As with self-driving cars, validation will take “millions of hours”, and more degrees of freedom mean a higher bar. “it’s the edge cases — the unexpected — that will ultimately set the bar”, and “safety is deeply grounded in a perception of what is considered to be acceptably safe — which is highly subjective.” In the home the bar is “stratospheric”. It is set by “perceptions and assumptions of safety — and perceived threats to what is considered sacrosanct — rather than hard evidence.” - Instinct over reason. A figure in the dark kitchen, or a robot watching from the end of the bed, is no physical risk. But the reaction is “driven by biology and instinct, not rational thought”, and so is hard to overcome. - Being human. “part of being human is having the autonomy to do the “boring” tasks that life is full of, rather than being relieved of these”. - Conclusion. Success depends on “the deeply complex challenge of ensuring human acceptance, rather than the merely complicated challenge of making them in the first place.” Companies should build teams focused on the societal challenges: “I’m just not sure they realize this yet.”

Concepts and frameworks. - Complex versus complicated challenges. - Acceptable safety as subjective and perception-driven. - Edge cases set the bar. - Perceived threats to what is sacrosanct. - Societal barriers as the decisive barriers to a technology’s success. - A Luddite-like backlash that succeeds. - The sci-fi feedback loop (Virk).

The logic matches his risk-innovation framing (societal and ethical factors as threats to an enterprise’s value that its developers overlook, as in his 2019 BMI “orphan risks” paper). But this post does not use those terms.

Analogies and comparisons. Self-driving cars: a literal comparison for safety validation. Earlier waves of industrial automation, and the Luddites (linking his 2023 post): historical and structural. Asimov and robot sci-fi: cultural drivers.

On AI. Robots with OpenAI-style language and learning abilities are general-purpose and adaptive, and that general-purpose character is what makes them different from past automation.

On risk. Job displacement, physical safety at the edge cases, psychological insecurity and the uncanny, and threats to the sacrosanct (home, children, pets). Risk here is explicitly about perception and value, not only measured hazard.

On companies and leaders. Figure and Tesla are vision-driven and overpromising (Musk’s “billion plus” robots). They are technically capable but naive about society.

On governance. Implicit only. Public acceptance and backlash act as de facto governance. He makes no regulatory proposal.

Change of view. A self-reported shift: from dismissing humanoid robots to taking the bet seriously, while doubting that it will succeed.

Quotes. - “But the biggest barriers are likely to be societal rather than technological.” - “safety is deeply grounded in a perception of what is considered to be acceptably safe — which is highly subjective.” - “the future success of humanoid robots will depend primarily on the deeply complex challenge of ensuring human acceptance, rather than the merely complicated challenge of making them in the first place.”

2024-08-11 — school-of-advanced-technology-transitions — “Envisioning a university-based School of Advanced Technology Transitions”#

Provenance. His own prose. It is an institutional concept note: first drafted briefly in November 2022, expanded in February 2024 (per his update note) and lightly updated for this post. It is written in a planning or proposal register and was republished while he prepared the IEEE keynote “The new science of advanced technology transitions”. There is no sign of AI drafting. The header image is MidJourney.

Argument in his terms. - Since 2022 “the gap between what we teach and research around advanced technologies and the reality we’re living in has widened considerably”. He knows of no concerted academic effort to close it. - The need, as a claim about novelty. “we are at a scientific and technological tipping point in human history”. AI, quantum, advanced gene editing, BCIs and automation are emerging at an accelerating rate and “confounding conventional thinking on how to ensure new technologies benefit society”. The resulting opportunities and challenges are “night and day different from those we’ve faced in the past”. “They are not navigable through conventional thinking, ideas, assumptions, education, and skills.” What is needed is new thinking, research, philosophies, skills, jobs and organisational structures, and working across expertise, sectors and communities. - Vision: technologies harnessed to “enrich and empower individuals, communities, and societies”, for “a future that is better than the past for as many people as possible.” - Research spans the specific (particular technologies) to the general (“models and theories that broadly address advanced technology transition”). It draws on philosophy, innovation theory, responsible innovation, technology ethics, political science, business, economics, communication and public engagement. It is “use-inspired” and “unbounded by conventional thinking”. - Degrees. A BS, a professional Masters and a PhD in Advanced Technology Transitions. Specialisations include “just and equitable technology transitions”, responsible innovation, public interest technology, governance and policy, and applied ethics. Informal learning runs through museums, YouTube and AI tools. - Role. The school would serve as “a guide or “pilot” for individuals, communities and organizations”, empowering stakeholders “to make informed decisions”. - He closes by inviting redesign, since “conventional ideas of what university schools, departments and colleges do and teach are likely to become increasingly irrelevant”.

Concepts and frameworks. Advanced technology transitions as a field (a “new science”). A tipping point in human history. Transitions that conventional thinking cannot navigate. Transdisciplinary, use-inspired scholarship. The school as pilot or guide. Just and equitable transitions. Responsible innovation and public interest technology as components.

Analogies and comparisons. None to specific past technologies. The implicit claim is discontinuity: the present differs “night and day” from past transitions.

On governance and who decides. Experts and scholars are cast as “thought leaders” and pilots who help stakeholders decide. Public engagement appears as a contributing discipline. The model is guide-and-empower rather than expert rule.

On education. Universities are failing to keep pace. Education, including lifelong and informal learning, is central to navigating transitions.

Change of view. He says the ideas are “probably more-so” relevant than in 2022.

Quotes. - “They are not navigable through conventional thinking, ideas, assumptions, education, and skills.” - “confounding conventional thinking on how to ensure new technologies benefit society” - “serving as a guide or “pilot” for individuals, communities and organizations”

2024-08-18 — four-ways-of-thinking-about-advanced-technology-transitions — “Four ways of thinking about advanced technology transitions”#

Provenance. His own prose, including a quoted excerpt from his own book Future Rising (2020), Chapter 48, “Boundaries”. The post reports the thought experiment from his IEEE ISCT keynote (Bali, 13 August 2024), illustrated with his home-made Lego-and-ribbon ladder videos.

Argument in his terms. - Pippard’s ladder (book excerpt). In 1980 Brian Pippard described “discontinuities”, which Maynard calls “the blindsides of the physical world”. These are abrupt, irreversible transitions whose onset is “all but impossible to predict”. Twist the four-rung ladder and it suddenly tangles, and reversing the twist does not undo it. He connects this to climate tipping points, nonlinear dynamics, and complex interconnected systems under stress. The excerpt ends: “unless we learn how to spot early warnings and stay clear of critical tipping points, we run the risk of, quite literally, crashing our future.” - New extension. A quadrant model built on two axes: “degrees of freedom” (restrictive versus open, with more options on the open side) and “mindset” (preserving things as they are versus embracing change). - Stated assumptions. We live in a closed system (the planet), and tipping points come from pushing against its boundaries. And: “It also assumes that we cannot simply turn off technology innovation, but instead need to find ways to manage and channel the inevitability of technological change.” He adds: “I suspect some will disagree with this latter assumption. But as change is fundamental to living within a dynamic universe, I’m comfortable with it.” - Avoid (bottom left). Stay well clear of tipping points. Common in climate policy. Benefits are weighed critically against harms, and “decisions are risk-averse and substantially informed by social factors” such as equity and wellbeing. Includes “policies and other governance mechanisms designed to slow innovation where the outcomes are uncertain”, proceeding cautiously, and anticipating impacts. He describes this neutrally; it is effectively the precautionary stance. - Adapt (lower right). Actively stabilise instabilities (clothes pegs on the ladder) to push tipping points further off. Analogous to climate adaptation, renewables, and (controversially) geoengineering. For technology transitions it means understanding instabilities triggered by advanced technologies, such as “the impacts of generative AI on learning for instance, or social cohesion”, and building resilience. The limit: “within a closed and constrained system the chances are that it just delays the onset tipping points rather than eliminates them.” - Extend. Refuse to accept apparent boundaries and extend them (the lengthened ladder). Examples: space (though “I’m a skeptic of the idea of other planets — most notably Mars — being a “plan B””), fusion, AI, bioengineering, quantum technologies and human enhancement. Historical precedents: steam power, synthetic fertilisers and the internet “redefined boundaries that previously constrained us as a species.” This too only postpones tipping points. - Embrace (top right). Accept tipping points and work through what lies on the other side. It is “the most challenging” and “quite perilous”. He admits: “to be honest I’m not sure what embracing a disruptive technology-driven tipping point might look like — and especially how it might play out with respect to who thrives and who does not”. He suggests that “technology driven tipping points are the norm rather than the exception in human existence” and that we are in “a stable patch”. So “we need to be developing the knowledge and insights necessary to ensure harm is minimized and benefits maximized” at the next irreversible transition. - Humility. “It may be so deeply flawed that it should be resigned to the trash can of bad ideas.”

Concepts and frameworks. - Pippard’s ladder, discontinuities, broken symmetry, and tipping points in apparently stable systems. - Irreversibility and hard-to-predict onset. - Early warnings. - A closed system with boundaries. - The degrees-of-freedom × mindset quadrant: Avoid / Adapt / Extend / Embrace. - Resilience. - The inevitability of technological change (“manage and channel”). - Distributive outcomes of transitions: who thrives and who does not.

Analogies and comparisons. Climate change: structural, underpinning Avoid and Adapt (adaptation, renewables, geoengineering). Historical technology revolutions (steam, synthetic fertilisers, the internet): literal examples of boundaries being redrawn. Space expansion. The ladder is a physical metaphor (conceptual).

On AI. One of the technologies that may “extend” boundaries, and also a source of instabilities (in learning and social cohesion) that call for adaptive resilience.

On risk. Framed as abrupt, irreversible systemic transitions that are hard to foresee. The risk lens is systemic, not about particular hazards.

On governance. Slowing innovation under uncertainty is presented as one legitimate quadrant among four, but his stated assumption that innovation cannot be turned off limits how far avoidance can go. He calls for new knowledge to manage embraced transitions.

Change of view. Explicitly signalled. Until now, Pippard’s ladder was a warning about tipping points to avoid (“This is vey much in line with my original thinking around the metaphor of Pippard’s Ladder”). The Embrace quadrant departs from this: he now entertains going through discontinuities deliberately, while admitting he does not know what that would look like.

Quotes. - “It also assumes that we cannot simply turn off technology innovation, but instead need to find ways to manage and channel the inevitability of technological change.” - “unless we learn how to spot early warnings and stay clear of critical tipping points, we run the risk of, quite literally, crashing our future.” (from his Future Rising excerpt) - “You could, in fact, argue that technology driven tipping points are the norm rather than the exception in human existence”

MEDIUM#

2024-07-10 — neuralink-update-july-2024 — “Neuralink update”#

Provenance. His own prose. Quick takeaways from Neuralink’s live update on X.

Argument in his terms. - Why the establishment pushes back. Neuralink does not play by the conventional medical-device rules. It is fast, visionary and good at storytelling, engages the public, and “brings a disruptive digital technologies mindset to a sector that tends to be anything but disruptive.” “Some of this is most likely justified”, and he links his own earlier concerns (his 2019 paper with Scragg and the March 2024 post). Yet “for all the hype and hubbub, I can’t help but be inspired by what Neuralink are achieving, and what they aspire to.” - Takeaways: - 3D multi-depth electrode arrays. - “Lifestyle integration”: a wireless, invisible implant that combines “the usability of a premium consumer product with the utility of an advanced medical device”. He thinks this is underappreciated and a likely “game changer” for adoption. - Optimus integration, including thought-only communication between “a human with an AI interface and an AI that looks like a human”. - Cybernetic enhancement beyond biological capability. - Musk’s “synergistic tech integration” across his companies. He describes this as a master plan (Mars and more) that draws on robotics, AI and “understanding and nudging human behavior”. It makes capabilities seem to appear from nowhere. - Musk is “still one of the most interesting and potentially transformative tech leaders around”. The post ends on an unresolved note: “Of course, whether this is something to be excited or concerned about is another question entirely”.

Concepts. Synergistic tech integration. The consumer-tech mindset versus medical-device conservatism. Human enhancement and cybernetics.

Analogies and comparisons. Consumer technology product design versus medical devices (literal).

On companies and leaders. Admiration for Musk’s integrative strategy and ambition, with concerns acknowledged but not developed. His tone toward a disruptive, rule-bending innovator is warmer here than his earlier critique of permissionless innovation might suggest.

Quotes. - “brings a disruptive digital technologies mindset to a sector that tends to be anything but disruptive.” - “Of course, whether this is something to be excited or concerned about is another question entirely”

2024-08-04 — 7-key-takeaways-from-elon-musk-and-lex-fridman — “7 key takeaways from Elon Musk’s latest conversation with Lex Fridman about the future”#

Provenance. His own prose. Musk and Fridman appear only through embedded video clips.

Argument in his terms. - Why cover Musk. Musk has “an outsized influence on how people think about and approach the future”. The recurring justification: “the important thing here is not how wild the ideas are, but how much influence they have over how the future unfolds.” - The seven takeaways, with his responses: 1. Altered humans. High-bandwidth BCIs and AI–human symbiosis. His critique: “the assumption that the speed of data transfer is synonymous with the speed of understanding transfer”. 2. Better than human. Enhancement framed as “fixing” people, which has a “Will Caster” (Transcendence) feel. It is naive and hyper-speculative, but the momentum is real, and it “should absolutely be part of a much wider conversation around where we collectively want to go as a species.” 3. Digitally altered states. He is sceptical of the “electrical signals are everything” model, but takes seriously the idea of the BCI as a general read-write device that modulates perception. 4. Safe AGI through BCI bandwidth. Musk’s plant analogy for alignment “made me smile”. He calls it sci-fi territory, but influential. 5. Training data from Teslas and Optimus. Serious privacy and security questions: “I’m not sure anyone’s asking serious questions about cars and future humanoid robots.” 6. A billion humanoid robots a year. Whether this happens depends on consumer acceptance, economics, regulation and geopolitics. Hence the need for students trained in “transdisciplinary approaches to supporting the emergence of societally beneficial technologies”. 7. Technology policy. Regulation as a “hardening of the arteries”: “This isn’t a new idea. But it is one that has growing political cachet in some tech quarters — and it isn’t wholly wrong.” He doubts Musk’s grasp of governance. If such figures put “their thumbs on the scales” of policy, “I hope that there’s an informed counterbalance.” - His overall stance. He enjoys the conversation: “I get where he’s coming from”, and suspects “we’d get on well”. Musk’s “egregious actions” frustrate him. The reason to listen is to be prepared “for where he and others aspire to take the future”.

Concepts. Influence over the future as a reason to take visions seriously. Data transfer versus understanding. The BCI as a general input/output device. Training-data privacy from embodied devices. Informed counterbalance in tech policy.

Analogies. Transcendence and Will Caster (conceptual, from his book).

On governance. Partial sympathy with the critique that regulation accumulates too much. Concern about tech leaders capturing policy. He wants an informed counterweight rather than rejecting their input.

Quotes. - “the important thing here is not how wild the ideas are, but how much influence they have over how the future unfolds.” - “This isn’t a new idea. But it is one that has growing political cachet in some tech quarters — and it isn’t wholly wrong.” - “I hope that there’s an informed counterbalance.”

2024-07-28 — massive-new-study-reveals-new-insights-into-ubi — “Massive new study reveals new insights into UBI”#

Provenance. His own prose. It includes quoted conclusions from two NBER working papers and excerpts from OpenResearch participant stories, which are not his. A section heading reads “The OpenAI Unconditional Cash Study”, evidently a slip for OpenResearch. He notes Sam Altman’s substantial funding without comment.

Argument in his terms. - The OpenResearch cash study (1,000 people at $1,000 a month for three years, against a $50 control group) found limited, tenuous significant effects on health and employment. He puts this down partly to “small signals in noisy data”. He asks whether the right indicators were measured: “what was measured than what was affected”. - The labour result. Recipients worked slightly less. He reads this as a sign that low-income US workers “are being pushed to work harder than they want”, and that cash gives them agency. - Care with evidence. He takes the advocacy organisation’s more positive agency findings “with a grain of salt”, because “when an organization is committed to showing certain outcomes, it’s very hard not to put a positive spin on ambiguous data”. He values the academic papers for being more measured. Qualitative stories are insightful but open to cherry-picking. - Against paternalism. On the fear that recipients lack self-control and need paternalistic oversight: “I find little if any evidence for this.” “most people have the ability to take actions that lead to a potentially better future for them, their families, and their communities, given the chance.” - Link to technology. UBI may be one way of giving people that chance, “especially as advanced technologies continue to transform the landscape around how people obtain and use the resources” they need.

Concepts. Agency. Anti-paternalism, which parallels the Choice Engines post from two weeks earlier. The limits of quantitative indicators (“not everything that counts can be counted”, a saying often attributed to Einstein). Advocacy bias in evidence.

Relevance. Inequality and justice, and the social response to technological disruption of work. A careful handling of evidence under uncertainty.

Quote. “most people have the ability to take actions that lead to a potentially better future for them, their families, and their communities, given the chance.”

2024-07-30 — watching-movies-in-class-technology-innovation — “Why all undergrads should take at least one course where they watch sci-fi movies in class”#

Provenance. His own prose, but the post is marked “(Updated May 2025)” (“Now in its eighth year”). The text as mirrored therefore reflects revisions up to May 2025, not only July 2024.

Argument in his terms. - He defends FIS 338, The Moviegoer’s Guide to the Future (created 2018), in which students watch twelve complete sci-fi films together in class. - Against the “efficiency model” of teaching. It puts quantity before quality: “It’s also a dangerous model in that it risks placing education before learning, and outputs (such as assignments and grades) ahead of outcomes (such as transformed lives).” It also excludes students who do not thrive in formal settings. - Understanding how technology is changing the world matters to “every single student”, because it means “equipping them to thrive in a future that is being transformed by advanced technologies like artificial intelligence and more.” - Watching together creates a “buzz” that sparks ideas and discussion reaching beyond class. The approach rests on a solid pedagogy underneath.

Concepts. Transformative versus efficiency-driven learning. Inclusive, low-stress learning environments. Sci-fi film as a way of opening up thinking about technological futures across disciplines.

Quote. “It’s also a dangerous model in that it risks placing education before learning, and outputs (such as assignments and grades) ahead of outcomes (such as transformed lives).”

LOW / NONE#

• 2024-06-25 — op-ten-emerging-technologies-2024 (low). His announcement of the WEF Top Ten Emerging Technologies 2024 from the “Summer Davos” in Dalian. He has contributed to the list since 2012. He admits “I must confess that I was worried that AI would end up dominating the technologies we selected” and is glad the list casts a wider net than digital and consumer tech (elastocalorics and others).
• 2024-07-03 — free-book-films-from-the-future (low). Offers free PDFs of Films from the Future (2018). He says the book is about how we think about and develop transformative technologies, not a guide to specific ones, and that “many of the ideas in the book are reflected in my current work around the future of being human and navigating advanced technology transitions”. This is a continuity signal.
• 2024-07-07 — sharing-space-cady-coleman (none). A warm review of former astronaut Cady Coleman’s book Sharing Space (his Mission: Interplanetary co-host). It covers inclusion in space and AstroAccess, and his observation that an Earth “disability” can become an “ability” in microgravity. Coleman’s passages are quoted.
• 2024-07-24 — exogenous-willpower-and-weight-loss-drugs (low; guest post). Written by Sean Dudley (ASU Chief Research Information Officer) on GLP-1 drugs as “exogenous” or “liquid willpower”, agency, medicalisation, and holistic health. The mirror’s byline lists Maynard, but the subtitle credits Dudley as guest contributor. It is not evidence of Maynard’s views, only of his choosing to publish work on technology-mediated agency.
• 2024-08-01 — elon-musks-new-primary-school-ad-astra (low). A news note on Musk’s Ad Astra school. It is an opportunity to rethink education “in a post-AI world” and for “navigating increasingly transformative advanced technology transitions”. But Musk carries “baggage”, and there is a risk of indoctrination: “a bold experiment in developing the thinkers and leaders of a future, or merely an incubator for mini-Musks.” He is fairly trusting that Musk’s enterprises “tend to be led by level headed executives”.