Late Lessons, Jensen Huang and AI
Markdown version

B01 digest: 2014-11-23 to 2018-08-22 (18 posts)#

This batch is the pre-Substack foundation. Most of it is Conversation op-eds and 2020 Science posts, later migrated. Several carry Substack dates months earlier than their true publication (see B01.md), so the chronology is roughly 2014 to 2018. All of it is Andrew’s own prose except the Robert Winston “Scientist’s Manifesto” repost. There are no Modem Futura posts in the batch. AI appears throughout, but always as one technology in a converging set, not yet as the centre of attention.

Main ideas#

1. Risk needs its own innovation. The central, most original idea of the batch is Risk Innovation, launched publicly in January 2016 from ASU’s Risk Innovation Lab (2016-01-11 thinking-innovatively…). His diagnosis: regulation and risk thinking are “built around previous technologies”. New things (IoT, “cloud-based AI”, wearables) get shoehorned into frameworks that don’t fit, and that mismatch itself increases risk and hides pitfalls. His remedy is “parallel innovation in how we think and act on risk”. It rests on two moves: - Risk as a threat to value. Value includes health, the environment and money, but also well-being, identity, culture, belief, dignity, equity and professional standing. - A “market” for risk. The constituents who hold something of value and are “willing to invest in protecting” it.

By mid-2016 (2016-03-02 WEF top 10) he adds two more elements: future value, meaning the risks of not developing a technology, and a distinction between risks open to tech fixes and subtler social and psychological risks that are not. In 2017 he applies the frame to SpaceX, where it becomes a question of social licence: will society “grant” the freedom to proceed (2017-04-10).

2. Responsible innovation as the governance baseline. He repeatedly endorses Stilgoe, Owen and Macnaghten’s four dimensions: anticipation, reflexivity, inclusion, responsiveness (2015-01-30; 2016-04-01). He places himself in a decades-old international community of science-technology-society, technology assessment and anticipatory governance scholars, and criticises elites who ignore that community: Schwab’s 4IR book “reads as if it were written in a vacuum”. His positive model is the 2016 DOT automated-vehicles policy. It is anticipatory, flexible and participatory, designed to evolve, and “a refreshing change from attempting to retrofit existing regulations”. Governance should neither brake innovation (“we can’t afford to slam the breaks”) nor let it run unexamined.

3. Complexity and systemic fragility. In the 2015 piece, the Indian grid blackout stands for converging technologies (AI, neuromorphic chips, robotics, the digital genome, gene editing) forming a “socio-techno-environmental system” that looks stable “until, suddenly, it isn’t”. Assessing risks one technology at a time, as was done for nanotech and synthetic biology, is insufficient. This systemic, transitions-level concern persists alongside his worry that “the gap between our technological capabilities and our ability to handle them responsibly has continued to widen”, which he voices while saying Bill Joy’s warning “still haunts me”.

4. Risk judgement is contextual and full-picture. In his nano and chemical cases (quantum dots and cadmium; Vantablack and carbon nanotubes), he argues against hazard-only alarm (“taken in isolation they are misleading”). He works through exposure across the life cycle and net risk trade-offs. But he equally insists that risks persist after public attention moves on. Nano risks “slip under the radar” because journalists don’t know what to ask and experts aren’t encouraged to speak, and the boundary organisations he served in (PEN) have closed. Caution is scaled to irreversibility and ignorance: gene drives demand being “exceptionally cautious” because with “life’s operating system” we cannot reboot.

5. Experts, institutions and publics. A strong early thread concerns: - researchers’ responsibility for hype (the press-release “chain of trust”) - scientific elitism (“revere, but not interfere”) - citizen-led science that empowers disenfranchised communities, with Flint as a case of experts and officials being wrong - public universities whose incentives crowd out public service (“the institution gets in the way”)

Risk is also who defines risk: “who considers what a risk (and to whom)”. Ordinary people “must have a say”.

6. Tech leaders: admired but socially naive. Musk (Tesla, SpaceX, nano “BS”) and Schwab are treated as visionary and often right on the technology, but deficient in social and political savvy. They face wicked problems, with GM foods as the example, and “numeric logic is often trumped by what we intuitively think and feel is important”. His prescription is partnership with the responsible-innovation and risk-innovation community (with openly acknowledged self-interest). He notes, without developing it, that private actors like SpaceX escape governance regimes written for states.

AI in this batch#

AI is a member of the WEF convergence set, not treated as categorically new. The risks he names: - 2015: autonomous weapons, and machines that don’t “respect human values” - 2016: AI inequality; “open AI ecosystems” that understand conversation and act on it, eavesdrop, and “independently decide what’s best for you”. This is a strikingly early framing of conversational, agentic AI as an autonomy problem. - Neural-lace brain-computer interfaces (BCIs) coupled to cloud AI (“Siri or Amazon’s Echo hardwired into your brain”) - The 2018 ten-risk Risk Bites list

The ten-risk list places existential risk from superintelligence as one item among many, alongside technological dependency and heuristic manipulation, the first appearances of the cognition and manipulation concerns that later become central. His tone is consistently anti-Hollywood: less “zombie apocalypse”, more Tay.

Cognition and formation#

The early node is the brain-tech piece (about September 2016). Neurotechnologies could “alter how someone thinks, feels, behaves and even perceives themselves and others”, without control or consent. He raises tDCS in classrooms and exams, TMS altering moral judgement, “cyber substance abuse”, and neuro-enhancement divides. It is the direct physical intervention version of the later worry about AI shaping minds through language.

Anything new or changed#

Most important posts for understanding his thinking#

  1. 2016-01-11 thinking-innovatively-about-the-risks-of-tech-innovation: the founding statement of risk innovation and risk as threat to value.
  2. 2016-03-02 how-risky-are-the-world-economic-forums-top-10…: threat-to-value applied; risks of not innovating; subtle vs tech-fixable risks; early conversational and agentic AI concerns.
  3. 2015-01-30 responsible-development-of-new-technologies…: complexity and systemic collapse; responsible innovation; the pro-innovation stance.
  4. 2016-01-11 the-fourth-industrial-revolution…: his intellectual community and toolbox, the capability–responsibility gap, and inclusion of ordinary people.
  5. 2016-02-01 we-dont-talk-much-about-nanotechnology-risks-anymore…: nanomaterial risk from the inside; risks outliving public attention; the missing boundary organisations.
  6. 2016-03-31 considering-ethics-now…brain technologies: technology acting on cognition and moral judgement; who defines risk; the public’s say.