Late Lessons, Jensen Huang and AI
Markdown version

B11 digest: 2023-09-04 to 2023-10-20#

What this batch is#

This batch covers seven weeks in autumn 2023. About a third of the posts are wrappers for the last five episodes of The Moviegoer’s Guide to the Future, in which he reads chapters of Films from the Future (2018). The rest are short commentaries, including pointers to his NSF comments on advanced technology transitions and a Nature Nanotechnology commentary with Sean Dudley, plus one post co-written with José Lobo. There are no Modem Futura posts. The Zombified podcast and Unplugged live streams are only announced; their content is not in the corpus.

Main ideas#

1. Who gets a say is the central governance question. The strongest thread is that experts and developers cannot see or decide alone: - The PCAST post calls for “intentional two-way engagement with key communities” and rejects the AI-insider claim that AI is too complicated for non-experts as hubris in a democracy (09-04). - The Man in the White Suit is his teaching parable of “myopically benevolent science” (09-08). - The nanotech post criticises AI development “driven by a small group of experts and companies who believe that they have all the understanding they need” (10-02). - Inside the university, students’ views on ChatGPT are more sophisticated than many faculty assume, so they must be in the conversation now (09-18, 09-20).

He names Participatory Technology Assessment, Public Interest Technology and NISE Net as tools.

2. Lessons from past technologies, used structurally. Nanotechnology and GMOs are the reference cases. He speaks from first-hand experience (NEHI co-chair, adviser to the Project on Emerging Nanotechnologies). The lessons are about process, not hazard: transdisciplinarity, the arts, humanities and social sciences, and broad stakeholder engagement as conditions of success. GMOs are “a masterclass in how naivety, hubris, greed, and a lack of broad engagement” can block progress. This is a pro-innovation reading: failing to engage kills good technology. AI’s stakes are “far higher” than those of nano or GMOs.

3. Advanced Technology Transitions becomes a research agenda. The NSF TIP comments (09-25) are the fullest statement yet of ATT: - It gives a formal definition of technology transitions and argues that we are at a “tipping point in human history”, with non-linear change at unprecedented rates, and AI foundation models have shrunk the timescale of social disruption “from years to months”. - A notable claim: potentially catastrophic failure modes are not just unaddressed but obscured by outmoded approaches to innovation. - The listed research domains read as his programme: historic failure modes and principles; a governance spectrum from public engagement through soft law and agile governance to hard law; equity; the arts and humanities as “modulators”; “novel theories, models, and approaches to risk” (the risk-innovation thread); foresight; misinformation; jobs; and learning.

4. Critique of techno-optimism and permissionless innovation. The Andreessen post (10-19) is his most direct critique of a tech leader so far. He names technological foreshortening: reducing the complexity of technological history to smooth upward trends, which hides “the pain and suffering in the detail”. His objection combines justice (“who decides who will suffer and who will thrive”) with systems reasoning: - the granularity of impacts; - “growing non-linearity” between technological cause and societal effect; - the limits of a finite planet (the elastic band); - uncertainty about whether we can always innovate out of problems caused by “previous good ideas”.

He grants Andreessen’s long-run progress point and says he “revel[s]” in technology’s potential. Sustainability, ethics, trust and responsible innovation are “critical components” of a good future, not its enemies; he defends that cluster, not the precautionary principle as such.

5. Being human, personhood and AI. The Creator post (09-28) extends his “future of being human” lens to machines: - He questions the reflexive framing of machines as tools or slaves. - He entertains that if human worth “transcends our biological “wetware,”” sentient machines might merit freedom and autonomy, while saying “We’re still a long way from such a future.” - He reads Skynet-style existential-threat narratives as projections of human fear of the other.

The Unplugged series (AI consciousness, AI symbionts, cooperation science) points the same way.

6. Writing, learning and formation. In the students post (09-20) he discloses: “I typically don’t use ChatGPT myself when I write”. His reason is that writing “reflects who I am, and to relinquish that to a machine would be to diminish myself”. He also credits students who came to college “to learn”. This is an early, personal version of his concern with how AI use shapes the self.

7. AI as a multi-edged, transformative technology (co-written). The SDG post with Lobo treats AI as “hype aside, a transformative technology”. It weighs risks (the water and energy footprint, a first in these notes; bias; inequity; labour displacement, including in the Global South; rebound effects) against the promise of augmenting combinatorial invention, and calls for UN-centred, multi-stakeholder engagement. Being co-authored, it is weaker evidence of his own views.

8. Storytelling as an engagement method. The Films from the Future reflection (10-08) presents science fiction as a non-preachy way to build the “interconnected understanding” responsible innovation needs across all stakeholders. It admits the book “was a failure” commercially, criticises colleagues who called popular writing “professionally embarrassing”, and restates irreversibility (“small missteps can have outsized and irreversible consequences”).

Concepts in this batch#

Advanced Technology Transitions (defined); failure modes obscured by outmoded approaches; tipping point and non-linearity; technological foreshortening (new); granularity of impacts; responsive and responsible innovation; permissionless innovation (critiqued); two-way public engagement; Participatory Technology Assessment; Public Interest Technology; myopically benevolent science; the honest broker (Pielke Jr.); gain-of-function risk and preparedness; personhood beyond biology; writing as self; automation versus augmentation (co-written); “could” versus “should”; irreversibility.

What is new or changed#

Most important posts#

  1. 2023-10-19 marc-andreessen-ditch-sustainability. Technological foreshortening; justice; non-linearity; his stance between techno-optimism and precaution.
  2. 2023-10-02 responsible-ai-lessons-from-nanotechnology. Nano and GMO lessons for AI; criticism of the closed circle of AI insiders; stakes “far higher”.
  3. 2023-09-25 building-a-better-futures-tough. The fullest statement of Advanced Technology Transitions and his research programme.
  4. 2023-09-04 why-public-engagement-is-so-important. Engagement, democracy and expert hubris; the AI “too complicated” claim rejected.
  5. 2023-09-20 what-do-college-students-think-about-chatgpt. Writing as self; students as stakeholders; his personal practice.
  6. 2023-09-28 the-creator-and-being-human. Personhood beyond biology; existential-threat narratives reframed.