B03 digest: November 2018 to June 2019 (14 posts)#

What this batch is#

These are the Medium-era posts that followed Films from the Future (November 2018). They include republished Conversation and OneZero pieces (one from 2015), extracts from the book and sci-fi listicles. All of it is his own prose, with no AI, guest or co-written text. Apart from two low-relevance posts (a presentations spoof and an Oscars list), they set out a fairly complete early statement of his position as a risk scientist turned scholar of responsible innovation. They are addressed mostly to tech companies, startups and investors.

Main ideas#

1. Risk needs rethinking, not just applying: risk innovation, threat to value and orphan risks. The 2018-12-13 post sets out a three-part toolkit from the ASU Risk Innovation Lab: - Risk innovation: “we need parallel innovation in how we think about and act on risk”. - Risk as a “threat to value”: risk as a threat to something of importance to a person, community or organisation. This lets in autonomy, dignity, trust, self-esteem, way of life and belief. It “extends conventional thinking rather than replacing it”. - Orphan risks: risks seen as too ill-defined, complex or irrelevant to attend to, yet able to derail an enterprise. He presents the term in 2018 as “a new one”. It is aimed at startups and funders, it sits largely outside regulation, and it is argued mostly on business grounds. In 2019-03-01 he extends it to consumer data (fitness trackers). Here orphan risks are one tool within a broader reframing of risk, not an AI-specific idea.

2. Chemical risk assessment as a template for “algorithmic risk” (2019-03-05). This is the most developed analysis in the batch. He argues by structure, not literally (“an algorithm is not a chemical”), that five concepts should transfer: hazard versus risk; exposure; consequences; exposure-response; and weight of evidence with checks and balances. He coins “algorithmic exposure” (anyone affected by an algorithm’s decisions is exposed) and “algorithmic exposure-response relationship”. The tone is that of an evidence-first risk professional. He warns against “knee-jerk reactions” to single startling studies, using the self-driving “predictive inequity” paper and Kate Crawford’s reading of it. He accepts that “there’s no such thing as zero risk” and that not deploying carries risk too. He also insists that discrimination, livelihood, liberty and dignity count as harms. His closing line ties past technologies to AI: we should not “repeat the mistakes of the past as we reinvent the future.”

3. Responsible innovation against both permissionless zeal and ethics-washing. In the 2015 piece republished in 2018-12-15 he defends Musk, Hawking and Gates against ITIF’s “Luddite” label. His argument is that caution is not smashing technology and that ITIF misses “the importance of innovating responsibly”. He cites the EEA Late Lessons from Early Warnings reports as a record of ignored early warnings (noted here only as his own citation). He criticises OpenAI’s founding logic as the belief that the answer to innovation is more innovation. “Permissionless” innovation appears in scare quotes as innovation driven by “power, wealth and a lack of accountability”. The 2019-04-15 post adds the operational half. Ethics principles and advisory boards (Google’s short-lived AI council) are necessary but insufficient, and at worst are “smoke-and-mirrors”. What is needed is standards, measurable expectations, enforceable checks and balances, culture, stakeholder buy-in (including affected communities and regulators) and long-term training.

4. AI: what it is and what worries him. AI appears mostly as “algorithms” and as one of several converging technologies, alongside atomic-scale engineering, DNA reprogramming and hyper-connectivity, that are “for the first time in human history” transforming what we can do. He calls himself “something of an agnostic” on superintelligence, which he treats as “rather speculative”. He also notes that superintelligence and the Singularity attract a near-religious fervour and provoke extreme reactions. The concrete risks he names are: - bias and “predictive inequity”; - crime prediction and being labelled good or bad by machines; - surveillance, such as DeepMind’s lip-reading system; - automation and inequality; - manipulation: AIs that learn “to use our cognitive vulnerabilities against us”, “machines that know us better than we know ourselves” and are unbound by human norms, and the need for tests of “when we are being played by machines”.

This thread comes through Ex Machina, which he returns to again and again. It is the clearest seed in this batch of his later concern with language, persuasion and cognition.

5. Who decides, and the role of publics. Leaving technology questions to “experts” is “an abdication of responsibility” (2019-03-06). The future is too often “designed by the powerful”, and design must be “de-marginalized” (2019-03-31, drawing on the Beatitudes). Technologies are “tacitly sanctioned by millions”, not steered by lone heroes (2019-04-26). He approves the gene-editing summit’s demand for “broad societal consensus”. Sci-fi films are “social and educational levelers”.

6. Stance and temperament. “Don’t Panic” but not complacent. He holds an obligation to innovate, and argues that renouncing technology from privilege denies choices to others. He is humble about how far complex futures can be designed, and he knows technology cannot deliver love or happiness. Sci-fi’s value lies in being untethered from accuracy, but it risks fostering dystopian fatalism. The recurring warning figure is the wealthy, unaccountable, overconfident founder, such as Hammond in Jurassic Park and Nathan in Ex Machina, set against the could/should line.

Concepts appearing#

Risk innovation; risk as threat to value; orphan risks; social risk landscape; hazard versus risk; algorithmic exposure and exposure-response; weight of evidence; no zero risk; predictive inequity (the paper’s term); responsible innovation; “permissionless” innovation (pejorative); operationalising ethics; could versus should; technological convergence; de-marginalising the future; AI exploiting cognitive vulnerabilities; superintelligence agnosticism; “Don’t Panic”; sci-fi as social leveler.

New or changed#

• Earlier roots. Much of this restates positions he already held. The Musk piece dates from 2015, the ethics course from 2013, and the posts link to 2015–16 Conversation and Nature Nanotechnology work on risk innovation and threat to value.
• Orphan risks. He introduces it in December 2018 as a new term. It is framed for business, not for AI.
• Algorithms-as-chemicals. This is a new move in this batch: an explicit transfer of his chemical and nanomaterial risk-assessment training to AI. His voice here is more technocratic and more cautious about evidence than in his later cultural writing. He guards against over-reacting to a single study as much as against neglect.
• Ethics boards. The ethics-board piece hardens his earlier view that “good intentions” are not enough into a specific criticism of AI-sector ethics-washing.
• No change of mind is signalled within the batch.

Most important posts in this batch#

1. 2019-03-05 · should-we-be-treating-algorithms-the-same-way-we-treat-hazardous-chemicals: his most explicit bridge between chemical risk science and AI.
2. 2018-12-13 · tech-startups-orphan-risks: where risk innovation, threat to value and orphan risks are first defined.
3. 2019-04-15 · tech-companies-need-an-ethics-reset (“Ethics Boards Won’t Save Big Tech”): operational responsible innovation and his criticism of AI ethics-washing.
4. 2018-12-15 · if-elon-musk-is-a-luddite-count-me-in: responsible innovation versus innovation at all costs, and his criticism of OpenAI’s founding logic.
5. 2019-03-06 · navigating-the-risks-and-benefits-of-new-technologies: the book’s closing statement of his temperament.
6. 2019-03-31 · design-principles-for-de-marginalizing-the-future: who decides, power and inclusion.