B13 notes: 2023-12-03 to 2024-02-05 (15 posts)#

Reading notes on Andrew Maynard’s Substack posts in batch B13. Only his own prose is treated as evidence. Quotes are exact, including his original typos and curly punctuation.

Batch context: these posts run from about a year after ChatGPT’s launch to early 2024. The period covers the UN High-Level Advisory Body’s interim report, the WEF Global Risks Report 2024 and Davos, the ASU–OpenAI partnership, Neuralink’s first human implant and the Apple Vision Pro launch. The posts are short-to-medium explainers and commentaries. Many are reactions to a document or event. None is a long original essay. No Modem Futura podcast posts appear in this batch (the podcast-related items are an index to his Moviegoer’s Guide to the Future audiobook-podcast, a write-up of an ASU Future of Being Human … Unplugged broadcast, and an event announcement that mentions the Mission: Interplanetary podcast).

Relevance summary:

HIGH#

2024-01-01 — the-future-of-being-human-in-2024 — “The Future of Being Human in 2024”#

Provenance. His own prose throughout (a New Year’s essay signed by him and dated January 1, 2024). The header image is Midjourney-generated. There is no AI-written text.

Argument in his terms. - The ASU Future of Being Human initiative was set up about 18 months earlier to address “what defines us as individuals in a technologically complex world”. The pace of AI change since then has made the question more urgent than anticipated. - The core worry is a tipping point. Technologies may stop being “a means of augmenting and expressing who we are” and begin to “fundamentally change who we are — or even what we are”. He calls this what “drives my work more than anything”. - AI as a mirror. Machines that mimic language mastery, creativity, thinking and reasoning have “held a mirror up to our understanding of who we are (albeit a rather imperfect one)”. They force the possibility that what we think of as defining us is “simply the product of biological computations that machines can, in principle match — or even exceed”. The same machines also have the power to alter what it means to be human. - Extrinsic versus intrinsic technologies. Most past technologies were extrinsic to our sense of self: they augmented or briefly modulated it. Few were intrinsic, changing us “from our constituent molecules and cells all the way up to our consciousness, self-awareness, beliefs, capacity for care and empathy”. Over coming decades, emerging technologies may let us alter the intrinsic “base code” of being human, “or will even lead to alterations in this base code without our agreement or permission”. - Language is part of the base code. AI systems with “increasing mastery over language” are part of this trend, because language is part of the ““base code” of self-identity, self-understanding, and social identity and understanding”. So are gene editing, neuroscience, brain–machine interfaces and “the intersection between AI and any technology that has an impact on how we think and behave”. - Convergence, not AI alone. He lists nanoscale science and engineering, quantum technologies, gene editing, synthetic biology, neuromorphic computing, neuroscience, BCIs, “and even social media”. Where these overlap, “new capacities for fundamentally changing who and what we are” emerge. - Open about outcomes. “How this will play out is anyone’s guess at this point.” Evolution may “trump technology at every turn”, or we may for the first time “break away from our biological roots”. - Firmness. He is strong on the importance of the question: “one of the most profoundly important questions we face as a species”. He is hedged on predictions: “many of the fears and aspirations that AI has sparked are more speculation than imminent reality”.

Concepts. The future of being human. The tipping point from augmenting who we are to changing what we are. Extrinsic versus intrinsic technologies. The “base code” of being human, with language as part of it. AI as mirror. Simulacra of the essence of being human. Technological convergence. Change “without our agreement or permission” (consent and agency in human transformation).

Comparisons. None to specific past technologies. The contrast is structural: past technologies were mostly extrinsic, and emerging ones may be intrinsic.

On AI. A technology that emulates what humans take to be uniquely theirs (language, creativity, reasoning). He expects further advances in 2024: “we’re getting perilously close to creating simulacra of the essence of being human within our machines”.

On AI risk. The risk is framed as identity and formation, not catastrophe: loss or alteration of what makes us human, possibly without consent. He treats hype on both sides (fears and aspirations) as largely speculative.

On cognition, language and formation. This is the clearest statement in the batch linking AI’s language capacities to human self-formation. Language is framed as constitutive of self and social identity, so a machine with language mastery can reach into that “base code”.

Quotes. - “our technologies begin to fundamentally change who we are — or even what we are.” - “emerging technologies will give us the ability to alter the intrinsic “base code” if you like of what it means to be human” - “AI systems that have increasing mastery over language — part of the “base code” of self-identity, self-understanding, and social identity and understanding — are part of this trend.” - “many of the fears and aspirations that AI has sparked are more speculation than imminent reality”

2023-12-22 — un-governing-ai-for-humanity — “Governing AI for Humanity: A Compelling Roadmap from the United Nations”#

Provenance. His own prose, with quotations from three sources: - the UN interim report; - his own 2008 WEF notes (sole-authored); - the 2010 proposal for a Global Centre for Emerging Technology Intelligence (written “with my colleague Tim Harper and other Council members”) and the 2018 WEF Global Future Council on Agile Governance report (a council product he contributed to).

The “Top Takeaways from the Interim Report” section is his own first-person commentary (“takeaways I personally had”). It is not an AI-generated takeaway box.

Argument in his terms. - Strong praise. The interim report is “perhaps, one of the most important AI governance analyses to have been published to date”. It was produced in two months without sacrificing quality. It is “neither a report that gets tangled up in speculative long term AI risks, nor misses the transformative significance of AI breakthroughs.” That is the balance he values. - A lineage he places himself in. He traces the report’s thinking to his own work: - At WEF’s first Summit of the Global Agenda Councils (Dubai, 2008) he developed the idea of a Global Institute on Emerging Technology Policy. - With Tim Harper, this became a proposal for a Global Centre for Emerging Technology Intelligence, presented at Davos in 2010. - Later, the WEF Global Future Council on Agile Governance (2018), where he was a member, defined agile governance as “adaptive, human-centred, inclusive and sustainable policy-making”. - He is careful about causality: “There’s no concrete evidence of causal links between these and the 2010 proposal, apart from my continued input.” - The 2010 proposal’s diagnosis (co-authored). A “new paradigm” was needed that “predicts and avoids potential hurdles”, engages multiple stakeholders, “identifies and addresses possible health and environmental impacts before they occur”, and “responds rapidly to new developments”. Yet there was a “gaping chasm” between knowing a new approach was needed and knowing what it should look like. This anticipatory, pre-harm framing grew out of his nanotechnology-era work and is re-presented here as the backstory to AI governance. - His takeaways. - Governance grounded in the UN Charter and human rights. - The “AI divide” between Global North and South. Global governance is needed so that “purely market driven AI development doesn’t, by default, favor privileged communities”. - AI and climate, and AI and the SDGs. Organisations focused on the SDGs but not on AI risk becoming “increasingly irrelevant”. - Public Interest Technology and AI for social good. This matters “despite the prevalence of assumptions amongst some AI developers that the market around transformative and disruptive AI will automatically deliver on social good.” - Multi-stakeholder collaboration: “I’m an ardent advocate for multi stakeholder engagement around advanced technology transitions.” - Agile AI governance. - The five guiding principles. - Conclusion. “AI for good cannot simply be left in the hands of tech entrepreneurs and AI experts, but is something that everyone with a stake in the future of humanity needs to be a part of.”

Concepts. Agile governance (defined as above). The Global Centre for Emerging Technology Intelligence (2010). The governance of emerging technologies as a field with a lineage of more than a decade. Advanced technology transitions. Multi-stakeholder engagement. Public Interest Technology. The AI divide. AI for social good versus market-driven innovation. Governance that must “reflect qualities of the technology itself” (quoting the report approvingly).

Comparisons. Emerging technologies generally, and implicitly nanotechnology: the 2008–2010 WEF council work dates from his nanotech period, and the 2024-01-14 post says the WEF councils shifted from a nanotech focus to emerging technologies around 2011. The comparison is structural: AI is treated as the latest of the “powerful platform technologies like AI” in a continuous governance problem, not a new category.

On AI. A transformative platform technology whose capabilities are outrunning governance: there is “a growing disconnect between the capabilities of powerful platform technologies like AI and effective governance mechanisms”.

On AI risk. He prefers near-to-mid-term, grounded risk and benefit analysis to “speculative long term AI risks”. Inequity (the AI divide) is a central harm.

On AI companies and leaders. Markets alone “do not and cannot” address humanity’s complex challenges. He criticises the assumption among “some AI developers” that the market will deliver social good. He says this without hostility to market innovation (“the report doesn’t discount the value or market driven innovation”).

On governance and who decides. Global, inclusive, networked, adaptive and multi-stakeholder. Anchored in human rights. The decision cannot be left to entrepreneurs and experts, and “you cannot develop AI for social good or in the public interest without stakeholders who represent these interests being at the table.”

Change of view. None. He explicitly presents continuity from 2008.

Quotes. - “This is neither a report that gets tangled up in speculative long term AI risks, nor misses the transformative significance of AI breakthroughs.” - “I’m an ardent advocate for multi stakeholder engagement around advanced technology transitions.” - “AI for good cannot simply be left in the hands of tech entrepreneurs and AI experts” - “despite the prevalence of assumptions amongst some AI developers that the market around transformative and disruptive AI will automatically deliver on social good.”

2024-01-14 — wef-global-technology-risk-trends — “WEF Global Technology Risk Trends: Looking Back Over 18 Years”#

Provenance. His own prose. It includes tables he compiled (images) from WEF Global Risks Reports 2006–2024, and one quoted WEF definition of “frontier technologies”.

Argument in his terms. - A qualified critique of the WEF Global Risks Report. He has followed it since near its start and is “regularly asked to contribute”. He calls it “in some ways, a flawed report”: it crowdsources expert views through surveys with pre-identified risk categories, so it “tends to reflect the current “risk perception zeitgeist” rather than over the horizon emergent risks.” He credits the analysis as “always sophisticated”, and the report as a useful “barometer” of expert concern. - His trend reading (self-described as “exceptionally crude … but … still insightful”). - 2006: convergence, electromagnetic fields, pervasive computing and nanotechnology appear. - Nanotechnology persists until 2013. He notes his role as chief science advisor to the Project on Emerging Nanotechnologies, which was “front and center of global discussions around the potential dangers of not developing nanotech responsibly.” Its persistence was surprising “at the expense of other disruptive emerging technologies”. - 2011: a shift to emerging technologies. He chaired the Global Agenda Council on Emerging Technologies that year. - 2016–2020: “adverse consequences of technological advances” appears, alongside the Fourth Industrial Revolution framing. - 2021: a shift to digital inequality, digital power concentration and “failure of technology governance”. He reads this as recognising “the importance of approaching sociotechnical systems from a risk perspective, rather than simply focusing on technologies alone.” It is disappointing that technology governance lasted only two years. - 2023: “frontier technologies” appears. - 2024: AI is named explicitly for the first time. - Conclusion. AI’s late appearance suggests “the risk domains are driven more by headlines than foresight.” The reports have moved from “a rather naive and limited perspective” to a more nuanced one. But the wisdom of the expert crowd “isn’t always capable of looking over the horizon”. He recommends adding foresight methodologies.

Concepts. “risk perception zeitgeist” versus “over the horizon” emergent risks. Crowdsourced expert risk perception and its limits. Sociotechnical systems as the proper unit of risk analysis. Failure of technology governance as a risk category. Frontier technologies. Advanced technology transitions. Foresight as a complement to expert surveys.

Comparisons. Nanotechnology (literal and personal: his PEN role and the headline-making nanotech risk debates of 2006). Electromagnetic fields from cell phones (a public health risk concern of the time). Orbital debris. Cyber attacks. Misinformation. The nanotech case is used literally, as risk-perception history. It is also an implicit lesson that institutional attention can lag and fixate.

On AI risk. The risk categories that matter are sociotechnical: inequality, power concentration and governance failure. He worries about risks that “if not mapped out, are the ones most likely to take us unawares.”

On governance and expertise. Expert crowds reflect the zeitgeist and regress toward what is in the headlines. Structured foresight is needed. Decision-makers otherwise “would be ill-prepared”.

Quotes. - “it tends to reflect the current “risk perception zeitgeist” rather than over the horizon emergent risks.” - “we were front and center of global discussions around the potential dangers of not developing nanotech responsibly.” - “the importance of approaching sociotechnical systems from a risk perspective, rather than simply focusing on technologies alone.” - “perhaps suggesting that the risk domains are driven more by headlines than foresight.”

2024-01-17 — ai-global-risks-2024-wef-davos — “Unpacking AI in the 2024 World Economic Forum Global Risk Report”#

Provenance. His own prose. It contains extensive quotation from the WEF Global Risks Report 2024, including two long “next global shock” call-out boxes in the addendum. Only his framing and evaluative sentences are evidence.

Argument in his terms. - Headlines versus the report. Davos headlines suggest “a global AI catastrophe”, but the report is “more nuanced and less sensationalist than you might think from media coverage.” - History of AI in the report. He traces it from a 2014 existential-risk call-out, through “massive and widespread misuse of technologies” (2015), “adverse consequences of technological advances” (2016–2022) and “adverse consequences of frontier technologies” (2023), to “adverse outcomes of AI technologies” (2024). In 2024 that category ranks 29th of 34 in the short term and 6th in the long term. He calls this “an inflection point in awareness around AI-related risks”. But “If anything, this report is a call to action on climate, not AI.” - Misinformation and disinformation is ranked first in the short term, and AI-generated misinformation is second as a likely 2024 crisis. His own gloss is that AI’s role will grow, “aided and abetted, of course, by powerful individuals and organizations that not only benefit from the manipulation and disruption this will produce, but actively use disinformation to persuade people that they are legitimate sources of truth.” - “Mundane” risks over AGI. AGI concerns are “only briefly touched on in the report — in contrast to some headlines”. What is prominent is “more mundane (but still highly disruptive) unintended consequences of AI”: - supply-chain chokepoints; - concentration of AI power in a few models and companies; - anti-competitive practices; - ethical corner-cutting by poorer economies trading data for access; - the AI divide; - AI in cyber warfare and autonomous weapons. - Weak recommendations. The report is “rather lackluster” on action, as expected of a crowdsourced report where “innovative and creative ideas are likely to be subsumed by regression to the mean of conventional thinking.” On AI literacy: “a lot of talk … but my sense is that there’s a lot less action”. The report’s AI-literacy proposal “barely scratches the tip of the iceberg around the role, purpose and impact of AI understanding and decision making across society.” - Addendum. Two “global shock” call-outs he found insightful: - Quantum: he thinks quantum technologies more broadly, not just computing, are the likely source of shock. - Unelected billionaires: “the outsized influence of unelected billionaires (something that’s been more than a little obvious over the past year or so to anyone using the platform formerly known as Twitter).” “As they say, watch this space …”

Concepts. Mundane-but-disruptive AI risks versus AGI speculation. Technological power concentration. The AI divide. AI-enabled misinformation as a tool of power (not just an error). Regression to the mean in crowdsourced expertise. AI literacy as necessary but far from sufficient. Convergence (AI amplifying quantum and synthetic biology risks, quoted from the report). Headline-driven risk perception.

Comparisons. Climate as the dominant global risk. The Twitter/X ownership as a live example of unaccountable private technological power. Quantum technologies.

On AI risk. Plural, sociotechnical and political. Misinformation, power concentration and inequity come first. Autonomous weapons are taken seriously. AGI is mentioned only as something the headlines overemphasise.

On AI companies and leaders. He is concerned about power concentrated in a few companies and in unelected billionaires. His Twitter aside is a pointed, if light, jab at Musk.

On governance and publics. Treaties and regulation are already being discussed elsewhere (he cites the UN report). The gap is public understanding and society-wide capacity for “AI understanding and decision making”.

Quotes. - “the focus on artificial intelligence in the risk report is more nuanced and less sensationalist than you might think from media coverage.” - “more mundane (but still highly disruptive) unintended consequences of AI” - “innovative and creative ideas are likely to be subsumed by regression to the mean of conventional thinking.” - “it barely scratches the tip of the iceberg around the role, purpose and impact of AI understanding and decision making across society.”

2023-12-15 — pope-francis-artificial-intelligence — “Pope Francis: Artificial intelligence ought to serve our best human potential and our highest aspirations, not compete with them”#

Provenance. A curated commentary. Most of the word count is block quotation from Pope Francis’s 2024 World Day of Peace message (not his). His own prose is the introduction, the section headings, one to three sentences of reaction per section, and the conclusion. His views appear in what he selects, how he frames it, and a handful of pointed sentences. (The post’s title is the Pope’s line.)

Argument in his terms. - He calls the message “a manifesto for socially responsible and beneficial AI that draws directly on what it means to be human”, and “one of the most thoughtful pieces on human-centric AI I’ve read in a while.” - Technology is not neutral. The idea of technologies being separate from how they are used (“people kill people, not technologies”) is “a fallacy”. “This complex intertwining of human nature and aspirations — and what we imagine and invent — is critical to better-understanding how the impacts of emerging technologies play out.” He endorses the “socio-technical systems” framing. - What AI is. He is pleased the Pope tackles definition, because “far too many people speak with authority on what AI might do without thinking about what it actually is”. He adds a signal of his own view: “I tend to go beyond the limitations of reproducing human abilities in how I describe AI”. That is, AI is not just imitation of human cognition. - Good intentions are not governance. This is his sharpest line: “I frequently come across the sentiment that good intentions will lead to beneficial AI, and that if tech leaders and innovators set out to “do good” all will be well with the world. Of course, this is sheer fantasy”. He welcomes the call for “bodies charged with examining the ethical issues”, that is, formal oversight. - Human uniqueness and what is worth protecting. On the Pope’s warnings about mortality and “technological dictatorship”, he concedes some will disagree. But “when approached from the question of what makes us human — what defines us — and understanding what’s worth protecting lest we discard it without realizing what we’re losing, these ideas resonate deeply with me and my own work.” - Cross-disciplinary dialogue and education. He has been calling for cross-disciplinary dialogue “for some time now”. Education for critical thinking about AI “especially resonated”. Getting universities to grasp the social and ethical aspects of technology is “so important — and yet so hard to achieve”. ASU’s School for the Future of Innovation in Society was set up for this, and “It’s been an uphill battle.” - Reservation. “You may not agree with all of it (I don’t).” He does not say which parts.

Concepts. Technology non-neutrality / sociotechnical systems. The “good intentions” fantasy. Formal ethical oversight bodies. Human dignity and human rights as evaluation criteria (endorsed via quotation). What’s worth protecting (what we might lose without realising). Cross-disciplinary dialogue. “algor-ethics” / ethics by design (the Pope’s terms, which he highlights). Critical-thinking-centred AI education. Socially responsible, human-centric innovation.

On AI. Poorly and variously defined (“a galaxy of different realities”, quoted approvingly). He resists defining AI only as the imitation of human abilities.

On AI risk. Selected emphases (via the Pope): job displacement with benefits to the few, lethal autonomous weapons, algorithms reducing persons to data, and “technological dictatorship”. His own emphasis is on losing what defines us.

On AI companies and leaders. Their self-professed good intentions are insufficient: “sheer fantasy”.

On governance. Formal oversight bodies. Many voices at the table. Ethics from the start of research through marketing (via the quote).

On education. Universities must teach the social and ethical dimensions of technology. It is hard, and his own institution has found it “an uphill battle”.

Quotes. - “Of course, this is sheer fantasy — but it’s a fantasy that a surprising number of people seem to have bought in to.” - “far too many people speak with authority on what AI might do without thinking about what it actually is” - “understanding what’s worth protecting lest we discard it without realizing what we’re losing, these ideas resonate deeply with me and my own work.” - “I tend to go beyond the limitations of reproducing human abilities in how I describe AI”

MEDIUM#

2024-01-30 — first-in-human-trial-of-neuralink-bci — “First In-Human Trial of Elon Musk’s Brain Computer Interface Begins”#

Provenance. His own prose. It includes quoted Musk tweets, the Neuralink website, the Washington Post, and one sentence quoted from his own 2018 book Films from the Future (chapter on Ghost in the Shell; the chapter PDF is attached). There is an update note at the end.

Argument in his terms. - Neuralink’s first implant is not the first BCI implant (at least 42 people have had them). What matters is the company’s “decidedly non-medical vision” of unlocking human potential, which makes it “a step forward to keep a close eye on.” - In 2018 he was “a little concerned by” what happens when “entrepreneurs and technologists become ever more focused on fixing what they see as the limitations of our biological selves” (quoting his book). - Neuralink’s early recruitment tagline, “No neuroscience experience is required”, may help explain its regulatory and ethical troubles. - Augmentation and norms. If Neuralink moves to augmentation in healthy people, it could disrupt “norms and expectations around human ability — including who gets to be be “augmented””, with knock-on effects on education and jobs: “(Imagine, for instance, a future where a BCI is a prerequisite for some jobs … or some educational programs)”. - Cult of personality. “Unlike pretty much any other medical device, company, or trial I’m aware of, Neuralink is, by proxy, caught up in a cult of personality”. For Musk’s fans the trials herald superhumans, “albeit superhumans that are locked into a subscription plan that governs what they can do and what they cannot.” Musk “doesn’t discourage this”, as the product name “telepathy” shows. - Hype check. “we’re decades — generations possibly — away” from what the followers imagine. There is “a gaping biological and technological chasm”. The medical promise is real. “we still have very little idea what the risk landscape will look like around devices that allow your brain to be wirelessly connected to the internet via your phone …” - Update (signalled change). He corrected a reference to people ““suffering” from quadriplegia” after Gregor Wolbring pointed out that it implies those not “human-typical” “are suffering and need to be fixed — which is clearly not right or appropriate.” This is a small but explicit correction toward disability-studies framing.

Concepts. Human augmentation versus restoration. Norms and expectations of ability, and who gets augmented. Cults of personality around tech leaders. Subscription-locked capability (commercial control over augmented bodies). An unmapped risk landscape for networked brains. Blurring boundaries between biology, machines and cyberspace (2018).

Comparisons. Other medical devices and trials (Neuralink is unlike them). Ghost in the Shell (conceptual).

On tech leaders. Musk is portrayed as encouraging hype. The company culture (“No neuroscience experience is required”) is linked to ethical lapses. Fandom is seen as distorting how a medical trial is understood.

On being human. BCIs could be a “game changer” for what it means to be human, with justice implications (access, prerequisites, subscription control).

Quotes. - “Neuralink is, by proxy, caught up in a cult of personality that is fixated on the idea that we can use technology to reinvent what it means to be human.” - “superhumans that are locked into a subscription plan that governs what they can do and what they cannot.” - “we still have very little idea what the risk landscape will look like around devices that allow your brain to be wirelessly connected to the internet via your phone …”

2024-02-05 — is-the-apple-vision-pro-a-social-game-changer — “Is the Apple Vision Pro a Social and Behavioral Game Changer?”#

Provenance. His own prose (first-person first impressions). It quotes a Stanford VHIL paper and Geoffrey Fowler (Washington Post).

Argument in his terms. - He queued for the headset and was “blown away — unexpectedly so”. That enthusiasm “doesn’t mean that we don’t need to be thinking hard and deep about how this tech might impact our lives, and how to successfully navigate the technology transition Apple is ushering in.” - Why it is different: video passthrough. Sci-fi (Ready Player One, Minority Report, Black Mirror and others) has “conditioned” us to expect seamless blending of real and virtual. Optical approaches (Google Glass, HoloLens) fell short. Apple instead “overlays real life over a virtual world”. Because the world the user sees is a digital reconstruction, “this reconstructed reality can now be manipulated in ways that physical reality simply cannot.” - Possibilities. Co-occupied remote spaces (twin cafés) could “shake up notions of travel, physical engagement, and even geographical boundaries”, and potentially “our sense of who we are”. - Concerns. - “after a generation or so, this technology could become addictively immersive”. It could be a game changer “both for what it allows people to do, and how it potentially messes us up if we’re not prepared.” - Psychological effects (via the Stanford paper): aftereffects, distance misjudgement, simulator sickness, and interference with social connection. - Privacy: the device “sucks up everything around it”. “we’ll need some recalibration of how we collectively think about privacy in the future.” He recalls Google Glass and the Steve Mann assault. - Hedged verdict. He is not yet convinced it is the social and behavioural breakthrough. It could be “Apple’s Google Glass”, or the start of a new era, “as we adjust as a society to living in a reality that is no longer constrained by … well … reality.” He describes himself as “still a IRL kind of person”.

Concepts. Technology transitions. Blended reality / spatial computing. Manipulable reconstructed reality. Addictive immersion over generational time. Social and behavioural game changers. Privacy recalibration. Sci-fi as a conditioner of expectations.

Comparisons. Google Glass (a failed social acceptance precedent, used literally). HoloLens. Quest 3. Neuralink (a direct neural feed “not going to see in my lifetime”). Waymo. Sci-fi films.

On cognition and formation. He is concerned with how immersive, mediated perception might reshape behaviour, social connection and self over time. There is a structural echo of his earlier concern with who controls the “shadows” we perceive: here, a reconstructed reality that can be manipulated.

Quotes. - “this reconstructed reality can now be manipulated in ways that physical reality simply cannot.” - “I can certainly see how, after a generation or so, this technology could become addictively immersive.” - “both for what it allows people to do, and how it potentially messes us up if we’re not prepared.”

2023-12-03 — 3d-artificial-brains-and-ai — “Are physical 3D artificial brains the next step in AI?”#

Provenance. His own prose: a 2023 introduction followed by the full text of his sole-authored 2014 Nature Nanotechnology commentary “Could we 3D print an artificial mind?” (9, 955–956). Treat the commentary as his 2014 thinking, re-endorsed in 2023. (The post’s “also available here” link points to a 2023 nanowire paper’s DOI rather than his commentary, a trivial error.)

Argument in his terms. - 2023 framing. The subtitle says “Data and algorithm based architectures have led to massive leaps in AI.” But the next step may be “a revolution in the substrates these run on”. Neuromorphic chips (IBM NorthPole, Intel Loihi 2), silver-nanowire networks and analog chips are developing, yet 3D neuromorphic architectures remain rare, and 3D-printed ones speculative. If manufacturable, they “would be a game changer”. The 2014 piece “is more relevant now than it was nine years ago”. He notes an omission: “the need for a secondary network to systematically adjust weights between neurons”. He admits some pride: the piece was once “so edgy” his editor hesitated, and now “it’s looking increasingly prescient!” - 2014 argument. - Brains are analogue, 3D and structure-dependent. Conventional 2D fabrication limits brain-like computing. - 3D printing converging with nanotechnology could plausibly (in 10–20 years) make dense 3D neuromorphic substrates with integrated power and microfluidic or heat-pipe cooling. He calls this “a naive thought experiment”. - Such substrates could be “a significant step towards realizing the as-yet science fiction imaginings of artificial minds”. Our only experience of massively interconnected 3D analogue processors is living organisms, whose emergent properties differ markedly from 2D processors. - It ends with a responsibility question: “how do we begin to think about responsibility in the face of such audacity?”

Concepts. Technological convergence (3D printing + nanotechnology + neuromorphic computing). Compute substrates as a driver of AI capability. Emergence (mind as an emergent property of complex 3D structures). Imaginable versus plausible (“shift from the fanciful to the plausible”). Responsibility in the face of audacity.

Comparisons. Nanotechnology (literal, his home field). The human brain (a structural and bio-inspired model, “inspired by, but not necessarily emulating”). The BRAIN Initiative and Human Brain Project.

On AI. Substrate matters: advanced AI, and possibly mind-like awareness, may depend on physical architecture, not just algorithms and data. This is continuous with his 2018 argument (see B08) that human-like machine intelligence would need radically different substrates. The 2023 subtitle implicitly acknowledges that large leaps came from data and algorithms on conventional hardware, and he now positions substrates as “the next step” rather than a precondition. He frames the direction of travel as “the race to develop ever more powerful AI”.

On AI risk. Only implicit: the ethical and responsibility questions of creating artificial minds with some awareness.

Quotes. - “we’re still only scratching the surfaces of compute substrates that could elevate AI to the next level and beyond.” (2023) - “how do we begin to think about responsibility in the face of such audacity?” (2014)

2023-12-20 — ai-superalignment-and-cooperation-science — “Does the key to AI superalignment lie in the science of cooperation?”#

Provenance. His own prose, summarising an episode of the ASU Future of Being Human … Unplugged broadcast (YouTube/audio) with Mark Daley (Western University) and Athena Aktipis (ASU, cooperation science). Several ideas are explicitly attributed to them: intelligence as an emergent property serving the system is “as Athena suggested”, and “Universe, at its most fundamental level, is information” is “Mark’s words”. The embedded video is not text evidence. Only his framing and his stated takeaways count as his views, and some takeaways are co-produced in conversation. (This is not a Modem Futura episode.)

Argument in his terms. - When OpenAI/Ilya Sutskever’s “superalignment” work (weak-to-strong generalisation) came up, he asked Aktipis how it resonated with biology. The insight “new to me” was the depth of parallels between superalignment and robust hierarchical cooperative biological systems. - In complex multi-agent systems, “less intelligent” agents can manipulate “more intelligent” ones to their own and the system’s benefit, through aligned interests and information flow. - Human exceptionalism blinds us. Viruses manipulate complex systems “without an ounce of what we might consider “intelligence””. The group toyed with “humans acting as metaphorical viruses in a world of superintelligent AIs”, though “I’m not sure how far anyone would want to push this analogy.” - Other takeaways listed: - honest versus dishonest information flow; - cooperative systems being hijacked (cancer); - threats building resilience and immunity, “a concept that suggests too much risk averseness is a bad thing”; - division of labour; - kombucha as a cooperative system. - Conclusion. Cooperation science has “a wealth of insights” for “beneficial development of transformative AI” when framed in terms of information, compute and aligned interests in hierarchical multi-agent systems, “irrespective of the “substrates” within which it resides”. There is “a compelling case for more collaboration — and even cooperation — between AI developers and cooperation scientists”. He notes counterpoints (parasitic behaviours, discussed in an earlier episode).

Concepts. Superalignment (OpenAI’s term, engaged seriously). Cooperation science. Hierarchical multi-agent systems. Substrate independence. Intelligence as emergent and not the pinnacle. Human exceptionalism as a blind spot. Hijacking of cooperation (cancer). Resilience through exposure (against excessive risk aversion). Cross-disciplinary serendipity.

Comparisons. Biology (conceptual and structural): cells, viruses, cancer, kombucha as models for AI–human systems.

On AI and AI risk. He takes superintelligent AI seriously enough as a framing to ask how humans might remain viable agents within such systems. His answer draws on interdependence and aligned interests, not on control. He is sceptical of human exceptionalism. The line “too much risk averseness is a bad thing” is a conversation takeaway, not a developed claim.

On AI companies. He engages OpenAI’s superalignment research respectfully, as a genuine research agenda.

Quotes. - “highly functional and complex multi-agent systems exist where what might be considered “less intelligent” agents manipulate the behavior of “more intelligent” agents to their benefit” - “there seems to be a compelling case for more collaboration — and even cooperation — between AI developers and cooperation scientists if we’re going to get AI right.”

2024-01-18 — asu-openai-collaboraton — “ASU announces a unique collaboration with OpenAI on using ChatGPT in education and research”#

Provenance. His own prose (short). It is an announcement-style post about his own university.

Argument in his terms. - He calls the ASU–OpenAI partnership (ChatGPT Enterprise for faculty and staff projects) “incredibly exciting”. It has “the potential to accelerate the transformative use of generative AI in academia and higher education.” - He wanted access himself (“myself included”). Cost and data privacy had been barriers. - He expects it “to unleash a tsunami of creativity and innovation”. A feedback loop with OpenAI “should be transformative”. - He acknowledges risk only briefly: “with the right checks and balances in place — and both ASU and OpenAI are being diligent here — this could be a truly catalytic collaboration.” - He hopes “more universities beginning to work with AI companies like OpenAI to build technologies and uses that transform how we learn, discover, and grow”.

On AI companies. He is warm and trusting toward OpenAI as a partner. He vouches for its diligence. This sits in some tension with the same batch’s warnings that tech leaders’ good intentions are “sheer fantasy” (2023-12-15), and that AI for good cannot be “left in the hands of tech entrepreneurs” (2023-12-22). Here the institutional partnership, with checks and balances, is presented as the responsible route rather than as a case of relying on good intentions.

On education. He strongly favours active experimentation with generative AI in teaching, research and administration, with students hopefully included later.

Quotes. - “I’m fully expecting this to unleash a tsunami of creativity and innovation” - “with the right checks and balances in place — and both ASU and OpenAI are being diligent here — this could be a truly catalytic collaboration.”

2024-01-21 — how-can-stories-unlock-pathways-to — “How can stories unlock pathways to positive futures?”#

Provenance. His own prose. It includes quoted dialogue from the short film The Assignment (by Tamara Krinsky and Taryn O’Neill; the line “Doom and gloom narratives remove your agency” is the film’s, not his), a quote from Tamara Krinsky, and a quoted passage from Chapter 29 (“Stories”) of his own sole-authored book Future Rising (2020), which is valid evidence of his thinking.

Argument in his terms. - He confesses: “I’m a sucker for a good dystopian sci-fi movie”, because dystopias “force us to think about what it means to be human when everything around us is falling apart”. Yet he was captivated by this non-dystopian film. - The film shows how the stories we tell affect our ability to influence the future. - His claim about persuasion. “Preach to someone about the future, and most people will shut down.” Beat people over the head and they fight or ignore you, “But tell them a story that resonates with them, and you open their mind to exploring possibilities that they would otherwise reject out of hand.” - He notes that all four featured authors are women from outside North America and Europe: “the importance of listening to diverse voices”. - Future Rising. Stories are “how we make sense of the world around us”. “stories become the pivot point between being able to imagine the future and beginning to build it.”

Concepts. Stories as the pivot between imagining and building futures. Narrative as an “on-ramp” to science (Scirens). Positive versus dystopian futures and agency. Diverse voices in futures thinking. Storytelling as a way past resistance.

On cognition and formation. Stories shape the mental maps people use to chart futures. This is an early, benign framing of narrative’s power to open minds. It is the flip side of his manipulation concerns: the same capacity to bypass resistance.

Quotes. - “tell them a story that resonates with them, and you open their mind to exploring possibilities that they would otherwise reject out of hand.” - “stories become the pivot point between being able to imagine the future and beginning to build it.” (from Future Rising, quoted by him)

2024-01-07 — the-future-of-being-human-is-analog — “The Future of Being Human is Analog”#

Provenance. His own prose. It describes a Vestaboard split-flap display installed at ASU’s Future of Being Human initiative, cycling through 800+ messages.

Argument in his terms. - He chose a deliberately low-tech, “intimately human” split-flap display to provoke reflection about the future of being human, because “In today’s technologically noisy world it’s easy to become so swept up in the capabilities we’re building that we lose sight of the essence of what it means to be human.” - It is “inevitable” that AI, genetic engineering and BCIs will change what it means to be human in some way, “Yet the essence of what makes us who we are is still more than the technologies we surround ourselves with.” - It is a “slow technology”, made to be savoured and to encourage reflection. “This is a technology that makes you think, rather than doing the cognitive heavy lifting for you.” - He is self-aware that this may be “my age and inclinations speaking”, but suspects human-scale technologies matter beyond his bias. He also worries, lightly, that “there’s simply not enough curiosity left in the world”.

Concepts. Slow technology. Human-scale technology. Technology that prompts thought versus technology that does “the cognitive heavy lifting”. The essence of being human as more than our technologies.

On cognition. A brief but early articulation of the contrast between technologies that support human thinking and those that substitute for it.

Quotes. - “This is a technology that makes you think, rather than doing the cognitive heavy lifting for you.” - “Yet the essence of what makes us who we are is still more than the technologies we surround ourselves with.”

LOW / NONE#

• 2023-12-06 — the-moviegoers-guide-to-the-future (low). A chronological index of his completed Moviegoer’s Guide to the Future podcast (the audio version of Films from the Future, 13+ hours). Promotional. He describes it as a journey “through emerging technologies and their responsible development and use”.
• 2023-12-13 — ais-using-lab-grown-brains-are-getting-closer (low). A short news note on a Nature Electronics “Brainoware” paper (brain-organoid reservoir computing), building on his earlier organoid-intelligence post. It pairs with the 3D-brains post on AI substrates. It closes by raising the consciousness question: “what happens if these bio-AI’s develop consciousness. And if they did, how we’d know …”
• 2024-01-28 — the-future-of-being-human-viktoria-modesta (low). An announcement of a live conversation with bionic pop artist Viktoria Modesta, recalling Mission: Interplanetary podcast episodes on disability in space. There is one relevant being-human idea: in zero-g “everyone starts from a point of “disability””, so norms of an “able” astronaut “are merely constructs of our own biases”.