Late Lessons, Jensen Huang and AI
Markdown version

B04 digest: July 2019 to July 2020#

The batch has eight posts. All are medium or high relevance, and none is a Modem Futura podcast post. Four are adapted from his earlier work: - chapters 3 and 4 of Films from the Future - a 2018 Astrobiology article - a 2019 JMIR commentary

Two are co-authored: a handbook chapter with Elizabeth Garbee, and an essay with Bas Boorsma. They are weighted accordingly. No AI-generated text appears. This is a consolidation period: the Risk Innovation Lab and Nexus apparatus is presented as established practice and applied to new cases (neurotechnology, astrobiology), alongside republished book material.

Main ideas#

1. Risk is about threats to value, and conventional risk analysis is not enough. The clearest statements are in the Mars post (2020-07-30) and the second Neuralink post (2019-11-01). “There’s more to risk than probabilities”: what counts is the kind of harm, its size, and who bears it. The losses people fear most are often unquantifiable: opportunity, hope, agency, dignity, justice. Risk innovation is defined in two ways: - by analogy with product innovation, turning creative ideas about risk into “frameworks and processes that people want to use”; - as recognising that emerging technologies produce a risk landscape so new that “conventional ways of thinking about risk are simply not up to the task.”

Failure often runs through how stakeholders “feel and act” rather than through technical systems.

2. Orphan risks, already in 2019. The second Neuralink post defines orphan risks as “often ignored, yet … frequently pivotal to an enterprise’s success or failure”. They are mostly hard-to-quantify social risks (ethical missteps, privacy, autonomy, social injustice), drawn from a fixed list of 18 used by the ASU Risk Innovation Nexus. The method has four steps: - start from value to enterprise, investors, customers and communities; - map the risk landscape; - look for clusters that “converge” into “blindsiding impacts”; - iterate.

In 2019 the frame is explicitly enterprise- and investor-facing. Ethics is recast as a condition of commercial success. This is a useful baseline for the 2026 orphan-risks work: the term is not new, but its audience and object may have changed.

3. A structural diagnosis of why good intentions fail. The responsible-innovation chapter (2019-08-13) names three dimensions of vulnerability in a fast-innovating world: - tight coupling: Perrow and Taleb, and the image of the unmuffled bell; - latency: harms surface more slowly than innovation cycles turn over, so later innovations amplify earlier harms; - value mismatch: market value is not societal value, which drives inequity.

Latency comes up again, more concretely, in the first Neuralink post: psychological harms might emerge long after people have become dependent on the technology. The connectedness essay (co-authored) carries the coupling argument into resilience: networks both build and undermine robustness, and dependence is a form of fragility.

4. Responsible innovation has to be culturally embedded, not only imposed. He values the aims of European responsible innovation and uses Stilgoe et al.’s anticipation, reflexivity, inclusion and responsiveness as a working framework. But he argues that in the US it has to grow from within innovator communities if it is to take hold. His models: - codes of conduct (the Responsible Nano Code) - Asilomar - DIY-bio norms - Debian’s constitution - “mutual worth creation” - culture change by design at ASU

Top-down regulation sets only “crude boundaries” and can push innovators toward workarounds. At the same time he quotes Sarewitz approvingly: leaving the risks and ethics of gene editing to experts is “wrong-headed, futile and self-defeating”. Two things are held together here: skepticism of top-down control, and insistence on inclusion and on responsibility beyond the expert.

5. Precaution, handled carefully. The Mars post sets the precautionary principle beside the IRGC framework and responsible innovation. He calls precaution “often misunderstood” and at the less rigorous end of the spectrum. But he endorses the UNESCO COMEST formulation (plausible, morally unacceptable, uncertain harm; proportionate response; a participatory process) as “a sound philosophy” for catastrophic uncertainty. In the first Neuralink post he warns against “paralysis by analysis” and against sensational dystopian speculation. He prefers “informed thinking about plausible issues”. His stance is between a caricatured precautionary ban and permissionless innovation.

6. AI as an opaque amplifier of old errors. AI appears mostly in the Minority Report chapter (2020-01-01), originally written in 2018. Machine-learning “criminality” classifiers continue phrenology, Lombroso and eugenics. The danger is not only bias (“the prejudices of its human instructors”). It is also opacity and lost understanding: we have “trained computers to do our thinking for us, but we no longer know how they’re thinking”. It is also pre-emptive injustice that denies human agency. His closing irony is that in trying to predict people we may build machines that behave badly “precisely because they are unpredictable”. Elsewhere, AI appears as one of several “inherently uncertain, potentially transformative” technologies (Mars), and as the eventual partner of brain-machine interfaces (Neuralink). The framing is ethical and epistemic. There is no existential-risk framing in this batch.

7. Mind, self and manipulation. The Neuralink posts treat the brain as the seat of identity. They raise: - the right to write to a brain - ownership of implants and subscription coercion - emotional manipulation by ads and news feeds wired in directly - enhancement producing a two-tier society

The behaviour-prediction chapter adds that neuroscience may make “our sense of self” look like “an illusion of our biology”. The bit-rot post extends this to identity over time: tech giants controlling how, or whether, we appear to posterity, and deepfakes eroding “a bedrock of reality”.

Past technologies and analogies#

Change or development signalled#

Most important posts in this batch#

  1. 2020-07-30 life-on-mars-astrobiology-and-thinking-differently-about-risk-4f5ab6a0cca9: the most compact statement of his risk philosophy: value, who bears the harm, precaution (COMEST), IRGC, responsible innovation.
  2. 2019-11-01 how-to-build-a-better-brain-machine-interface-while-not-falling-at-the-first-hurdle-cc238836a2b7: defines risk innovation and orphan risks (2019 version), and the landscape and cluster method.
  3. 2019-08-13 responsible-innovation (co-authored): tight coupling, latency, value mismatch; culturally embedded responsible innovation; codes of conduct and Asilomar; the Sarewitz caveat on expert self-governance.
  4. 2020-01-01 the-science-of-predicting-bad-behavior-2e095e9b3bcc: his early, sustained treatment of AI: bias, opacity, pre-emption, the non-neutrality of science.
  5. 2019-07-23 neuralinks-technology-is-impressive-is-it-ethical-812afb38b19e: can versus should, plausible versus speculative risk, latency of psychological harm, neural manipulation and inequality.