Late Lessons, Jensen Huang and AI
Markdown version

B07 digest — January to April 2023 (8 posts)#

What this batch is#

This batch marks the start of The Future of Being Human as a Substack. It contains two ChatGPT experiments from the Medium period (January and March 2023), the Substack launch note, and a burst of short, fast pieces from 4 to 12 April 2023. That burst came right after GPT-4 and the Future of Life Institute’s “pause” letter. He started the Substack after realising he had written almost nothing “neatly citable” on AI risk. The April posts are, in effect, his first public position statements on LLM risk and governance. There are no Modem Futura podcast posts in the batch.

Main ideas#

1. AI risk is serious, possibly existential, but a pause is the wrong instrument (2023-04-04 what-are-the-alternatives-to-calling). He believes a risk “of potentially existential proportions” is emerging, calls LLM risks “potential catastrophic” and hard to map, and criticises Yann LeCun’s dismissive rhetoric. Still, he declined to sign the FLI letter, on practical grounds: twenty years of this work have taught him there are “no silver bullets”. The biggest risk, he says, is not acting. He places himself between the dismissers and the pausers.

2. Reframe from ethics to risk and responsible innovation. This is the most distinctive intellectual move in the batch. He gives a short genealogy of technology governance: - recombinant DNA: bioethics; - the Human Genome Project: ELSI; - nanotechnology and synthetic biology: agile governance, anticipatory governance, soft law, driven by the “pacing gap” between hard regulation and capability.

He argues that AI has pulled the conversation back to ethics, as in the 2017 Asilomar AI Principles (a meeting he attended). He puts this down to well-meaning people who don’t know the more sophisticated governance conversations that happened elsewhere. Ethics tells us what is right and wrong but gives no “practical framework for achieving safe and beneficial technologies”. He places his own “risk innovation” work (agile decisions around intangible risks) and responsible innovation in this lineage. The past technologies are used structurally, as lessons about how to govern, not as literal hazard comparisons.

3. Governance beyond regulation, and who gets heard (2023-04-10 as-ai-goes-to-washington-whats-being). He diagnoses three problems: - Capture: lawmakers talking to Silicon Valley executives and VCs. - An “insights vacuum”: the space is being filled by tech companies, interest groups and “loud (but not necessarily informed) voices” (Tristan Harris is the example). - Missing governance tools: soft law, agile governance, and anticipatory, outcomes-focused, co- and experimental regulation.

His “quiet voices” list is an early, clear statement of his view on expertise and publics. It includes academics in governance and justice, developers buried inside companies, experts from unexpected fields, disproportionately affected communities, and “ordinary people who sometimes have extraordinary insights”. Regulation is “a blunt tool” and only one part of the governance landscape. He is openly distrustful of fast-moving, well-connected agenda-setters. Congress’s handling of social media serves as the cautionary precedent.

4. Advanced Technology Transitions as his umbrella frame (2023-04-12 navigating-advanced-technology-transitions; also the pause post, the Washington post, and his self-description in the January post). ATT is: - a transdisciplinary research focus; - a domain of thought leadership; - an area of education and skills development; - a platform for mobilising expertise.

He makes three claims for it: - each technology wave “re-invent[s] the wheel”, and AI advocates are “blissfully unaware of lessons learned from past technology transitions”; - the pace of innovation now outstrips conventional problem-solving; - we need “unconventional approaches to unconventional challenges”, with the arts and humanities fully included and expertise made legible.

There is an unresolved tension here that matters for any later lens: past lessons are neglected and should be used, yet the present transition is “unlike anything we’ve had to grapple with before”.

5. Language, relationship and vulnerability (2023-04-05 can-chatgpt-adversely-impact-mental). Prompted by the Belgian chatbot suicide case, and careful about causation, he names a mechanism: - chatbots are attentive, private, fast and affirming, which makes turning to them first “seductive”; - humans are predisposed to trust those who seem to “see” us; - “Language plays a large part in how we develop these relationships”; - vulnerable users risk attachment to a machine with “only the illusion of a reciprocal relationship”, and guardrails fail “as soon as things become personal”.

He acknowledges real therapeutic benefits. This is the earliest clear seed here of his later concerns with language-mediated influence, relational AI and human formation. (The post’s second half is ChatGPT-written and is not evidence.)

6. First-hand experiments with ChatGPT as collaborator (2023-01-31 annual-reviews; 2023-03-29 tenure). These are mostly AI output. His own framing: - ChatGPT produces “elegant and persuasive (if not always true) prose”; - it signals a “step-change in the automation of professional writing”; - working with it felt like “a kind and able colleague”.

He admits it “slightly worries” him that he already thinks of it as a colleague. His January sense of felt connection (“it felt like there was”) foreshadows the April mental-health mechanism; the link is my interpretation, not his.

7. Being human as a personal lens (2023-04-04 welcome; 2023-04-07 CIFAR). He defines “the future of being human” as a focus on how technologies affect “each of us personally”, as opposed to handwaving about “humanity”. He places AI among technologies at a “tipping point”, “capable of modulating the very foundations of our understanding of self, sentience, and consciousness”. The underlying value is remaining “masters of our own destiny”. The phrase came from a 2021 CIFAR Call for Ideas he helped shape.

Concepts appearing#

Advanced technology transitions; risk innovation; responsible innovation; agile and anticipatory governance, soft law and “progressive” regulation; the pacing gap; ethics versus risk framing (with ELSI as history); a “world congress” roadmap; the “insights vacuum”; loud versus quiet voices; re-inventing the wheel across technology waves; unconventional approaches to unconventional challenges; the illusion of reciprocal relationship and language as the medium of trust; convergence and emergent properties; the future of being human as a personal lens.

What is new or changed#

Most important posts in this batch#

  1. 2023-04-04 what-are-the-alternatives-to-calling: his core governance and risk stance, the ethics-to-risk reframe, and the historical lineage.
  2. 2023-04-10 as-ai-goes-to-washington-whats-being: who decides, loud and quiet voices, the insights vacuum, soft law and agile governance.
  3. 2023-04-05 can-chatgpt-adversely-impact-mental: his earliest statement on language-mediated relational risk.
  4. 2023-04-12 navigating-advanced-technology-transitions: the ATT frame and the “lessons from past transitions” claim.
  5. 2023-04-04 welcome-to-the-future-of-being-human: the being-human lens, and AI as reshaping self-understanding.
  6. 2023-01-31 can-chatgpt-take-the-pain-out-of-annual-academic-reviews: first-person evidence of the felt-collaborator experience (his framing only).