Late Lessons, Jensen Huang and AI
Markdown version

B12 digest: 2023-10-24 to 2023-12-01#

What this batch is#

This batch covers five weeks in autumn 2023: the Biden Executive Order on AI, the Bletchley Park summit, OpenAI’s custom GPTs, the Cruise robotaxi crisis, Sam Altman’s firing and return, and ChatGPT’s first anniversary. Of the 17 posts, five are high relevance, six medium and six low. The low posts are two guest essays by Brad Allenby, three ASU livestream notices and a Claude-generated timeline of the Executive Order. There are no Modem Futura podcast posts.

Main ideas#

1. AI risk has to be rethought from first principles, and with humility. The central post is 2023-11-26, Why everything you’ve ever heard about AI risk is wrong. Maynard starts from the textbook definition of risk as the probability of harm from an action, process or situation, and breaks it into five elements: - Cause and effect: “no cause, no risk”. Hazard and speculation, such as “AGI going rogue”, are not risk without a causal pathway. - Magnitude. - Harm, defined by social values. From this he concludes that risk is “ultimately a social construct”. - Time. - Perception, meaning biases, heuristics, and threats to identity, belief and meaning.

AI takes each element “to a whole new level”: we know almost nothing about the causes, effects, causal threads or timescales, the harms of not developing AI, or whether regulation could itself be a risk. His prescription is about process: humility, dropping egos, transdisciplinary collaboration and agility. He admits his own title was “a little hubristic”.

2. He tests the chemical paradigm on AI instead of importing it. The 27 November addendum is the batch’s most direct engagement with his chemicals and toxicology background. He sets out risk = Fn(hazard, exposure), stresses non-linear dose-response (thresholds, hormesis, low-dose effects), and maps it onto AI: - Hazard could range from “disrupting financial services” to “influencing human behavior”. - Exposure could range from AI agency over critical systems to “hints of ideas encountered over hours of social media use”.

The comparison is structural and exploratory; he finds no framework yet for AI hazard, exposure or transfer function. He also refuses to treat “zero exposure — as in no AI” as the default strategy, a clear rejection of abstention-style precaution that fits his symmetric framing of “the risks of going too fast, the risks of not going fast enough”.

3. Risk as a threat to value, orphan risks and the Risk Innovation Planner are brought to bear on AI companies. In two posts he points his long-standing ASU Risk Innovation work at AI developers. - 2023-11-15 reposts his 2018 article, re-aimed at AI through the failure of Meta’s Galactica. The article defines orphan risks: ill-defined, complex, seemingly irrelevant social risks that derail enterprises and are rarely governed by law. It also defines risk innovation as “parallel innovation in how we think about and act on risk”. - 2023-11-21 role-plays OpenAI’s board with ChatGPT working through the two-page Planner. Most of that text is ChatGPT’s and is not evidence of his views.

His own additions: - the distinction between value (worth) and values (right and wrong); - the claim that the framing is “agnostic to particular worldviews”, yet makes firms see how they threaten what others value; - a broad definition of “community” that includes people disadvantaged by not using a product.

The tools are aimed at enterprises and appeal to competitive advantage and survival. This is one strand among several, and the concept goes back to at least 2018.

4. The AI transition is socio-technical and cannot be left to AI experts or big tech. This theme runs through the White House, Altman and one-year posts. - ChatGPT’s impact came less from the technology than from its “social and technological landscape”; the launch was “a social and commercial as much as a technological step”. - “our AI future cannot be left solely to AI experts”. He calls for expertise in intelligence, consciousness, neuroscience, philosophy, personhood, ethics and responsible innovation. - He criticises “suck it and see” experimentation in a bounded, non-linear world where naive trials can push past “hard-to-spot tipping points”, as ChatGPT’s launch irreversibly did.

5. Governance: welcome state action, warn about capture and conventional thinking. He calls the Executive Order a serious “start”, but naive, citing among other things “an over-emphasis on following the lead of large tech companies and their agendas” and “a lack of evidence for learning from past technology transitions”. The US, he says, has resisted responsible innovation because of “an ethos of go fast, break things”. Leading firms have shaped regulation in ways “that seemingly favor commercial leaders”, and Altman was probably policymakers’ “shiny person”. On the Frontier Model Forum’s safety fund, he warns against “outmoded models of risk management” and wants novel conceptions of risk, the arts and humanities, and reimagined public participation. The Allenby guest posts he published (the “cognitive ecosystem”; against an IPCC for AI) are Allenby’s ideas, though he links the first approvingly.

6. AI in learning, and AI as a risk reducer. Generative AI can “flatten the learning distribution curve” by serving students at both edges, as inclusion rather than selection; he treats ChatGPT’s unreliability as a spur to critical questioning, with “persuasive incorrectness” as a caveat. On Waymo he does careful comparative risk analysis, checking the insurer’s human baseline against federal crash data and asking whether humans are the right comparison: “not all technologies — or companies — are created equal.”

New or changed compared with earlier batches#

Most important posts for understanding his thinking#

  1. 2023-11-26 everything-youve-heard-about-ai-risk-is-wrong. First-principles risk, the hazard-exposure paradigm applied to AI, no zero-exposure default, humility as method.
  2. 2023-11-29 the-year-that-generative-ai-changed-the-world. The socio-technical account of the ChatGPT transition, tipping points against “suck it and see”, and “cannot be left solely to AI experts”.
  3. 2023-10-30 white-house-goes-all-in-on-responsible-ai. Responsible innovation and his critique of the Executive Order’s naivety, including deference to big tech.
  4. 2023-11-15 navigating-orphan-risks. The 2018 statement of orphan risks, risk innovation and risk as a threat to value, re-endorsed for AI.
  5. 2023-11-21 ai-and-risk-innovation. The Risk Innovation Planner and value versus values, applied to OpenAI (his framing prose only).
  6. 2023-11-18 sam-altman-openai-impacts. Three frames (technology, governance, society), industry influence on regulation, and the race dynamic.