B13 digest: 2023-12-03 to 2024-02-05#

What this batch is#

Fifteen posts from the ten weeks around the turn of 2024, mostly reactions to documents or events: the Pope’s World Day of Peace message, the UN AI Advisory Body’s interim report, the WEF Global Risks Report and Davos, the ASU–OpenAI partnership, Neuralink’s first implant and the Apple Vision Pro. The rest are shorter pieces on AI substrates, a cooperation-science broadcast, stories, a slow-tech installation, and promotions. No Modem Futura material appears. The Pope and WEF posts rely heavily on quotation; only his framing counts there.

Main ideas#

1. Being human is the governing frame, and language sits at its core. His New Year post (2024-01-01) is the batch’s clearest statement of what drives his work. He fears a tipping point where technologies stop “augmenting and expressing who we are” and begin to “fundamentally change who we are — or even what we are”.

He introduces a distinction between extrinsic technologies (most past ones, which augment or briefly modulate the self) and intrinsic ones that could alter the “base code” of being human, possibly “without our agreement or permission”. AI’s “increasing mastery over language” is named as part of that trend, because language is part of the base code “of self-identity, self-understanding, and social identity and understanding”.

AI is also a mirror, suggesting that what defines us may be computation machines could match. Yet most AI fears and hopes are “more speculation than imminent reality”. The same concern appears in: - the Pope post, on “what’s worth protecting lest we discard it without realizing what we’re losing”; - the analog installation post, on technology that “makes you think, rather than doing the cognitive heavy lifting for you”.

2. Governance: agile, multi-stakeholder, public-interest, and not left to entrepreneurs. The UN post places AI governance in a lineage he helped build: - 2008: WEF notes on a global emerging-technology policy institute; - 2010: a Davos proposal (with Tim Harper) for a Global Centre for Emerging Technology Intelligence, calling for a paradigm that identifies health and environmental impacts “before they occur”; - 2018: the WEF council’s definition of agile governance as “adaptive, human-centred, inclusive and sustainable policy-making”.

He treats AI as the latest platform technology in a continuous governance problem. Governance should be global, rights-anchored and multi-stakeholder; markets will not deliver social good or equity (the “AI divide”); and “AI for good cannot simply be left in the hands of tech entrepreneurs and AI experts”.

The Pope post adds his bluntest line: the belief that tech leaders’ good intentions will yield beneficial AI is “sheer fantasy”. Formal oversight bodies are needed.

3. Risk as sociotechnical, plural and over the horizon. In the two WEF posts, crowdsourced expert surveys capture a “risk perception zeitgeist” rather than “over the horizon emergent risks”, and AI’s late arrival suggests categories “driven more by headlines than foresight”. Foresight should augment “the wisdom of the expert crowd”. He welcomes the 2021 shift to sociotechnical categories (inequality, power concentration, “failure of technology governance”).

His nanotechnology history is used literally: he was PEN’s chief science advisor when nanotech risk made headlines, and nanotech long crowded out other technologies in the report.

On AI he backs the emphasis on “more mundane (but still highly disruptive) unintended consequences” over AGI: misinformation as a tool of power, concentration in a few companies and “unelected billionaires”, the AI divide, autonomous weapons.

He praises the UN report for not getting “tangled up in speculative long term AI risks”. He criticises “regression to the mean of conventional thinking” in crowdsourced recommendations, and says AI literacy proposals barely scratch “the tip of the iceberg”.

4. Tech leaders and companies: sharp on personalities, warm on partnership. Neuralink is “caught up in a cult of personality” that promises superhumans “locked into a subscription plan”. The early “No neuroscience experience is required” tagline is linked to its ethical troubles. At the same time, the ASU–OpenAI post is openly enthusiastic (“a tsunami of creativity and innovation”) and vouches that both parties are “being diligent”. The implied reconciliation is institutional (partnership with “checks and balances” versus reliance on founders’ good intentions), but the contrast is worth recording.

5. What AI is: substrates, biology and emergence. He re-presents his 2014 Nature Nanotechnology commentary on 3D-printed neuromorphic “artificial minds”. The 2023 framing says data and algorithms produced the big leaps and substrates may be “the next step”. This implicitly updates his 2018 view that human-like AI needs radically new substrates (substrate is now the next frontier rather than a precondition). He closes on “responsibility in the face of such audacity”. The organoid post raises machine consciousness and “how we’d know”.

The cooperation-science broadcast engages OpenAI’s “superalignment” through biology: less intelligent agents can steer more intelligent ones in aligned hierarchies, and human exceptionalism is a blind spot. (In the Pope post he says he describes AI as more than the reproduction of human abilities.)

6. Formation through mediated reality and stories. The Vision Pro post flags a reconstructed reality that “can now be manipulated in ways that physical reality simply cannot”, warns of addictive immersion “after a generation or so”, and calls for “recalibration” of privacy norms.

The stories post (with a Future Rising excerpt) treats narrative as “the pivot point between being able to imagine the future and beginning to build it”. It also holds that stories open minds that preaching closes. This is the benign face of the capacity he elsewhere fears in manipulation.

Concepts appearing#

Extrinsic vs intrinsic technologies; the “base code” of being human (language included); AI as mirror; what’s worth protecting; agile governance; emerging technology intelligence; multi-stakeholder engagement in advanced technology transitions; public interest technology; the AI divide; the “good intentions” fantasy; sociotechnical systems; risk perception zeitgeist vs over-the-horizon risk and foresight; mundane-but-disruptive AI risks; technological power concentration; slow technology and cognitive heavy lifting; compute substrates and responsibility for audacity; superalignment via cooperation science; human exceptionalism; cults of personality and augmentation norms; manipulable reconstructed reality; stories as the pivot between imagining and building futures.

New or changed#

• Being human. The extrinsic/intrinsic distinction, and the explicit placing of language inside the “base code” of identity, are the most developed statement yet in this reading of how AI bears on human formation. They extend the earlier concern with language-based manipulation (B08) toward identity and self-understanding.
• Governance lineage. He makes his 2008–2018 WEF emerging-technology governance history explicit and ties it to AI. Nanotech-era anticipatory governance becomes the backstory for AI governance.
• Speculative risk. He is more openly impatient with speculative long-term risk narratives and headline-driven risk perception. He favours sociotechnical, power and equity framings.
• Correction. One explicit correction: he dropped “suffering” from quadriplegia after Gregor Wolbring’s challenge.
• Tension. Enthusiasm for the ASU–OpenAI partnership sits alongside his critiques of tech-leader self-regulation.

Most important posts#

1. 2024-01-01 the-future-of-being-human-in-2024. Intrinsic/extrinsic technologies, the language “base code”, his core motivation.
2. 2023-12-22 un-governing-ai-for-humanity. His governance philosophy and its lineage from nanotech-era WEF work.
3. 2024-01-14 wef-global-technology-risk-trends. Risk methodology (zeitgeist vs foresight) and nanotech history.
4. 2024-01-17 ai-global-risks-2024-wef-davos. His AI risk priorities: mundane, political and power-related over AGI.
5. 2023-12-15 pope-francis-artificial-intelligence. Non-neutrality, the “sheer fantasy” of good intentions, and what’s worth protecting.
6. 2024-01-30 first-in-human-trial-of-neuralink-bci. Tech-leader personality cults, augmentation norms and justice.