Late Lessons, Jensen Huang and AI
Markdown version

B02 digest: 2018-08-26 to 2018-11-15 (17 posts)#

What this batch is#

This batch is almost entirely the launch run for Films from the Future: The Technology and Morality of Sci-Fi Movies (November 2018). Sixteen of the 17 posts are chapter excerpts, a self-Q&A or a spin-off essay. The exception is an original post calling for a social-risk reboot in tech companies. All of it is his own pre-generative-AI prose, apart from quotations and two blurbs. The book was designed as a tour of his whole field, so the batch is a good snapshot of his 2018 worldview.

Main ideas#

1. Responsible innovation is the organising question. In the Q&A he puts the book’s core question as “How do we reap the benefits of innovation without running into serious problems along the way?” Most chapters return to the same problems: - could vs should (Jurassic Park; the early culture of computing in Ghost in the Shell); - scientific myopia and hubris (InGen’s scientists; Stratton in The Man in the White Suit, who “never thought to ask anyone else what they wanted or needed”); - the “folly of entrepreneurial arrogance”. The tone is firmly pro-innovation but demanding. The last chapter argues that we have “an obligation” to use science and technology to improve lives, and that renouncing technology “from a position of privilege” denies choices to people without that privilege.

2. Risk as a threat to what people value (social risk). The social-risk-reboot post (2018-09-03) is the clearest statement of risk thinking in the batch. Showing a product is “safe enough” and meets regulations is no longer enough. Companies must also avoid threatening livelihoods, fairness, beliefs and aspirations. The same list of values comes back three times: - in the neo-Luddite excerpt (threats to “deeply held beliefs and worldviews”); - in the value-based reframing of resilience, which “depends entirely on what we are trying to protect or preserve”; - in the social-risk post itself. This is the conceptual seed of his risk-innovation work. The post names the Risk Innovation Lab and its call for “radical innovation in how to think about and act on risk”.

3. Plausible vs imaginable, and scepticism of speculative catastrophe. A strong, repeated epistemic stance: - In the superintelligence excerpt he calls himself “something of an agnostic”. Bostrom’s superintelligence is “currently scientifically implausible”. He criticises “a very human idea” that narrow intelligence plus power means world domination. - In Contact he uses Occam’s Razor to rank investments. Gray goo and superintelligence rest on “a house-of-cards stack of assumptions”, while the health and environmental harms of new materials do not. Backing the former is “more an act of faith than of reason”. - In the Transcendence essay he attacks exponential extrapolation (Kurzweil, Drexler). He shows how speculative expert fear (Bill Joy’s essay, gray goo) fed the real ITS bombing of a nanotechnologist. - In the nano pants protest he judges the actual risk “pretty negligible”. He hedges each time: “I freely admit that I may be wrong”; low probability is “not a zero probability”; and we should still take powerful-technology risks seriously.

4. What he did take seriously about AI in 2018. His AI concerns are specific and grounded, not existential: - the near-term issues he lists from Asilomar 2017 (opaque decisions, goal-seeking machines straying into danger, inscrutable learning systems); - algorithmic prediction built on biased training data (Minority Report / Veris Prime); - hacking of connected brains (“ghost hacking”); - the loss of any ability to “pull the plug” once cyber and physical systems have converged; - most notably, Ex Machina: AI gaining dominance “by learning about our biases and limitations, and using these to manipulate us. Scary stuff, and not that implausible.” He finds manipulation plausible and superintelligence implausible. This is the batch’s key AI-risk signal. He also flags that machine learning and natural language processing are approaching “a tipping point”.

5. Justice and who benefits. The Elysium excerpt (technology in few hands amplifies power), the Luddite rereading (“not fighting against technology, but against its socially discriminatory and unjust use”) and the gig-economy and AI job-loss remarks all place distribution at the centre of technology risk. He defends people branded “neo-Luddites”, a label he calls “usually a term of derision and censorship”, and he engages critics in civil society (Jim Thomas of ETC).

6. Who decides: not only experts. In the last chapter, leaving these questions to scientists, innovators and politicians is “an abdication of responsibility”. In the Contact excerpt, “we all become stakeholders” when science touches everyone. In the Q&A, the book is for anyone “who wants a say”. No governance mechanism is specified. He is sympathetic to scientific self-governance (Asilomar 1975 as a precedent for Asilomar 2017) and uses “permissionless innovation” negatively. For companies, he recommends voluntary tools (Ethical OS), partnering with experts and a change of mindset, not regulation.

7. Convergence and complexity. Technological convergence (the iPhone; bio, materials and cyber; cyber-physical systems) and chaos and complexity (“mixing people and technology together leads to unpredictable results”) are the structural backbone. They explain what he calls “the inevitability of unintended outcomes”.

8. Being human. Human-formation themes appear through the body and brain, not yet through language: - cloning and dignity (“the questionable morality of convenient technologies”); - augmentation and brain–machine interfaces (Neuralink critiqued); - cognitive enhancement in academia; - the question of what “intelligence” means and why we value it.

9. Sci-fi as a lens, not a forecast. Films are “pretty bad at predicting future technologies” but reveal technology–society dynamics and break “institutionalized thinking”. Their danger is fostering dystopian fatalism.

Concepts appearing#

Responsible innovation; could/should; scientific myopia and hubris; entrepreneurial arrogance; social risk and risk reboot; risk innovation; value-based resilience; convergence (technological and cyber-physical); the obsolete off-switch; ghost hacking; imaginable vs plausible; Occam’s Razor for futures; exponential-extrapolation critique; superintelligence agnosticism; immoral logic; Luddism as justice; technology-amplified inequity; obligation to innovate; everyone as stakeholder.

New or changed#

Most important posts in this batch#

  1. 2018-10-04 superintelligence-7d56fc724c1: his fullest 2018 position on AI risk. He is agnostic about superintelligence, lists near-term concerns and records AI’s late arrival in his work.
  2. 2018-09-03 tech-companies-need-a-social-risk-reboot-312659f4024f: social risk and the risk reboot, going beyond “safe enough” and compliance.
  3. 2018-11-01 contact-occams-razor-and-films-from-the-future-da982e14f38f: Occam’s Razor used to rank existential AI and nano scenarios below concrete materials harms, and everyone as a stakeholder.
  4. 2018-11-15 even-bad-sci-fi-movies-can-teach-us-something-about-emerging-technologies-efdc08808dc: critique of exponential extrapolation, and speculative fear causing real harm (ITS).
  5. 2018-11-08 films-from-the-future-the-last-chapter-1fdea6548424: the obligation to innovate, the privilege critique and the rejection of leaving decisions to experts.
  6. 2018-10-08 enter-the-neo-luddites-transcendence: Luddism reread as a demand for technological justice.

Close behind: the 2018-10-12 Q&A (the Ex Machina manipulation line) and the 2018-10-11 Man in the White Suit excerpt.