Batch B04 notes: 2019-07-23 to 2020-07-30 (8 posts)#

All eight posts were read in full. None is a Modem Futura podcast post; the batch has no AI-generated text. Provenance problems here come from co-authorship (two posts) and from posts adapted from his earlier books and papers (four posts). Each post’s provenance is given below.

Relevance summary:

2019-07-23 · neuralinks-technology-is-impressive-is-it-ethical-812afb38b19e#

“Neuralink’s Technology Is Impressive. Is It Ethical?”

Provenance: His own prose. It first appeared on OneZero (Medium) and was republished here. It ends with an update noting his JMIR commentary (see 2019-11-01).

The argument: He takes Neuralink’s July 2019 launch seriously as technology: “This isn’t vaporware”. But he warns that companies like it risk getting “so wrapped up in what they can do, that they lose sight of the ethics behind what they should do.” The brain is “where the roots of our sense of self and identity lie”. So neurotechnology raises ethical questions and also social risks. It could change collective behaviour, disrupt norms and undermine values. He deliberately rejects dystopian speculation (“brain-hacking or mind-jacking”) as “rarely helpful”. He also warns against “paralysis by analysis”. What he asks for instead is “informed thinking about plausible issues”. He organises the issues into three areas: 1. Physiological impacts. He is “reasonably confident” that regulators and researchers can handle these. He draws directly on his own years working on the health risks of nanoparticles, while noting that regulators “have to be open to new ideas”. 2. Psychological and behavioural impacts. He calls these “more tricky”. Scrutiny must rise as uses move “from remediation to enhancement”. He flags latency: long lag times between widespread use and the appearance of psychological harm, together with dependence before the long-term effects are known. 3. Broader societal impacts. These include who owns the implant and the data, and who has “the right to write data to your brain”. They also include subscription and upgrade coercion and exposure to hacking. He raises manipulation through neural “write” capability: “ads that trigger an emotional response, news feeds that can manipulate how you feel”. Finally, a “two-tier society” of enhanced and unenhanced people.

He concludes that unless these questions are addressed early, either brain-computer interfaces “create more problems than they solve”, or Neuralink goes bust for not taking them seriously. The remedies he offers are early, “wider, deeper, and more informed” conversations and his own Risk Innovation Accelerator.

Concepts: “can” vs “should”. Ethical and responsible innovation. Plausible versus speculative risk. Remediation versus enhancement. Latency of harm. Ownership and “write” rights over the brain.

Past technologies: Nanoparticles, used literally as the source of his confidence about physiological risk assessment. Cochlear implants and deep brain stimulation serve as precedents that allowed “breathing space” for ethics. Iain M. Banks’ “neural lace” and Films from the Future supply the science-fiction framing.

AI: Only indirect. Neuralink’s long-term aim is an internet-connected overlay so users can “interface with future intelligent machines”, which he calls “very overtly aimed at changing society”.

Companies and leaders: He is even-handed toward Musk. He praises Musk’s “familiar model” of bringing together talent from different fields, and credits him with opposing neural adverts. His criticism is of the can/should gap, not of Musk personally. He frames responsibility partly as being in the company’s interest.

Cognition and formation: The brain as the seat of self. Mood, personality and memory altered by app. Emotional manipulation through direct neural write.

Quotes: - “there’s an urgent need for informed thinking about plausible issues, and how to navigate them” - “This could spell disaster if people become dependent on the technology before the long-term impacts are fully understood.” - “We might be looking at a future where mandatory auto-updates rewrite your hardware as well as your physical mind.”

2019-08-13 · responsible-innovation#

“Innovating responsibly in a culture of entrepreneurship”

Provenance: Adapted from chapter 32 of The International Handbook on Responsible Innovation (von Schomberg and Hankins, eds.), co-authored with Elizabeth Garbee. It is written partly in his first person: he taught entrepreneurial ethics at Michigan, and he writes “As I wrote in 2015”. It closes with “we would argue”. Treat it as joint text written in Maynard’s voice, with somewhat lower evidential weight than his sole-authored posts. Its concepts match his other work closely: risk innovation, worth and value, the nano references. It has 7,289 words.

The argument: - US entrepreneurial culture shares the ideals of responsible innovation (RI). His Michigan students wanted to “make the world a better place”. - But it rejects RI’s European, academic, top-down manifestations. Students found them “too academic, too institutionalized and too out of touch”. - He contrasts European RI with US values. European RI works through democratising the “governance of intent” and “right impacts” anchored in the European Constitution (Owen et al.). In the US, “right impacts are often contested” and government direction of innovation is “close to anathema”. - Unconstrained entrepreneurialism is dangerous because of three dimensions of vulnerability: - Tight coupling: Perrow’s Normal Accidents and Taleb’s Black Swan. He uses the image of a muffled bell of innovation that now “reverberates loudly round the globe at the speed of light”. - Latency: the innovation cycle becomes shorter than the time it takes harms to appear. Later innovations then amplify the impacts of earlier ones. - Value mismatch: market value versus societal value, which drives inequity; “the value of expediency is not the value of net societal benefit”. - Good intentions without codified approaches “remain good intentions, and no more”.

His proposed alternative is culturally embedded, bottom-up responsible innovation, backed by “external frameworks and influences”: - A framework of mutual worth creation from his Michigan course. Entrepreneurs create worth aligned with their own values, but succeed only by creating mutual worth with stakeholders. - Stilgoe et al.’s four dimensions of responsible innovation (anticipation, reflexivity, inclusion, responsiveness), recast as entrepreneurial practices: customer discovery, pivoting, failing fast. - Codes of conduct, especially the Responsible Nano Code (2008). - The 1975 Asilomar meeting on recombinant DNA. - DIY-bio community norms. - The Debian Project’s constitution and social contract. - Michael Crow’s value-driven redesign of ASU as a model for designing culture.

He also notes a clash of responsibilities. Entrepreneurs owe duties to investors, employees and family, and “going out of business unnecessarily is irresponsible”. Heavy toxicology testing that bankrupts a company could itself count as irresponsible.

How firmly he holds this: Firmly on the diagnosis. The solutions are offered as “a small number of examples from an as yet largely unexplored solution space”.

Governance and who decides: Top-down governance creates only “crude boundaries”. Durable responsibility has to be ingrained in communities. But an ethos alone is not enough. He quotes Sarewitz approvingly: leaving the risks, benefits and ethics of gene editing to experts is “wrong-headed, futile and self-defeating”. So he holds two positions at once: community self-governance, but not expert self-governance in place of public deliberation. Regulation can backfire, because imposed rules may push entrepreneurs “to find easier pathways”, or to redefine their goals.

Past technologies: Nanotechnology (the Responsible Nano Code), used structurally as a model for sector codes. Recombinant DNA and Asilomar, as the canonical and still-cited precedent. CRISPR and gene editing. Open-source software. His own 2015 Nature Nanotechnology line that early disregard of societal concerns locks technologies “into development trajectories that are highly susceptible to failure”.

Companies and leaders: Broadly sympathetic to entrepreneurs’ motives. He is critical of a culture with a “willful disregard for future consequences”, of treating regulatory loopholes as opportunities, and of the market as “ultimate arbiter”.

Education: University entrepreneurship programmes and his own ethics teaching, with ASU as an example of culture change.

Quotes: - “This is a culture of entrepreneurialism that, paradoxically, reflects the ideals of responsible innovation, yet rejects many of the manifestations of these ideals.” - “the good intentions of entrepreneurs will in many cases remain good intentions, and no more” - “the value of expediency is not the value of net societal benefit”

2019-09-04 · how-to-ensure-our-digital-legacy-isnt-lost-to-the-future-f6a226bc6792#

“How to Ensure Our Digital Legacy Isn’t Lost to the Future”

Provenance: His own prose. It includes a personal anecdote about a CD-ROM of his 1990s research files.

The argument: He adopts Vint Cerf’s “bit rot” and mentions Cerf’s “digital vellum” proposal. We “live under the illusion of information longevity and retrievability”. What matters is not storage but the ability to read and interpret. He opens with the 1992 Waste Isolation Pilot Plant team designing 10,000-year warnings for nuclear waste. He uses it conceptually, as a model of long-horizon communication through durable media, not as a point about nuclear risk. Written records outlast digital ones.

His worries: - Tech giants and proprietary formats control access and curate our digital footprint. “Who determines how we appear to future generations, or even whether we appear at all?” - In a world of deepfakes, “how will we establish a bedrock of reality” if it has been bit-rotted away?

The remedies he suggests are modest: being intentional about storage, open standards, pen and paper, durable artefacts. His tone is measured. He grants that “None of these scenarios are likely”, at least for the foreseeable future.

Relevance: Medium. It is an early statement of concern about big-tech control over memory, identity and the record of reality, and about human self-understanding across time. It is not about risk governance.

Quotes: - “we live under the illusion of information longevity and retrievability” - “unless we are happy for the tech giants to control and bury our digital legacy, we need to think more carefully about bit rot”

2019-10-10 · the-science-and-ethics-of-cloning-abeb41f1e5ad#

“The science and ethics of cloning”

Provenance: His own prose, adapted from chapter 3 of Films from the Future (2018) and pegged to Gemini Man. It quotes the 1997 “Declaration in Defense of Cloning” and the 2005 UN Declaration. Those quotations are not his views.

The argument: He explains Dolly and somatic cell nuclear transfer. The concept is simple, “biology rarely is”. He sets out the contested ethics: the UN Declaration was non-unanimous (84 for, 34 against, 37 abstaining), and there is a split between reproductive and therapeutic cloning. He gives a sympathetic but critical reading of Raël’s Yes to Human Cloning. With “the ‘I talk to aliens’ bits removed”, it reads like Kurzweil or Musk. It appeals to those who see “humans as no more than sophisticated animals and technology as a means of enhancing and engineering this sophistication”.

He notes that transhumanist ideas (disposable bodies, mind uploading) are “increasingly garnering mainstream attention”. Therefore “we have to take the possibility of human reproductive cloning seriously”. That includes the ethics of how we treat clones. He engages the Declaration’s claim that “traditionalist and obscurantist views” should not obstruct science, and points to the irony that it is easiest to find inside a mystical treatise. He does not take a firm position on whether reproductive cloning should be allowed.

Concepts and threads: Transhumanism and techno-utopian visions, which he links explicitly to Kurzweil and Musk. Can/should. Science fiction as a way into public imagination: it shows “how deeply this technology is embedded in our collective psyche”. AI and nanotechnology appear only in his description of Raël’s advocacy.

Quotes: - “while the concept of cloning is pretty straightforward, biology rarely is” - “we have to take the possibility of human reproductive cloning seriously”

2019-11-01 · how-to-build-a-better-brain-machine-interface-while-not-falling-at-the-first-hurdle-cc238836a2b7#

“The Many Ways Elon Musk’s Neuralink Could Go Wrong”

Provenance: His own prose. It summarises Maynard and Scragg (2019), JMIR 21(10):e16321, an invited commentary published alongside Musk and Neuralink’s paper. The risk-landscape figure comes from that joint work.

The argument and a signalled change of approach: He says he did not want to write “another commentary” on ethics (compare 2019-07-23). Instead he applied risk innovation: “Rather than critique brain-machine interface technology on ethical grounds, we asked what the orphan risks landscape might look like.” Ethics and social responsibility are reframed as conditions for enterprise success, protecting value to “innovators and investors, as well as users and the communities they are a part of”.

Definitions he gives: - Risk innovation: emerging technologies are creating a risk landscape so different from the past that “conventional ways of thinking about risk are simply not up to the task of navigating them”. It is also “the creation of value through creative approaches to potential dangers and pitfalls”, by analogy with innovation as value creation. - Orphan risks: “often ignored, yet are frequently pivotal to an enterprise’s success or failure”. They are often “hard-to-quantify social risks”: ethical missteps, threats to privacy and autonomy, and social injustice. The Risk Innovation Nexus has “a list of 18 of them”. - Method: start with value (to the enterprise, investors, customers, communities). Ask which orphan risks threaten it. Map a risk landscape. Look for risks that cluster and converge “in ways that increase the chances of truly blindsiding impacts”. Iterate as the landscape shifts.

Findings: Novel health impacts and “black swan” events call for organisational agility. The densest cluster of orphan risks lies in “organizations and systems”: internal values and culture, evolving regulation, and standards. So the biggest threats may be inside the company and its governing bodies. Constituency risks include ethics, social injustice (widening the rich/poor gap), privacy, autonomy, norms around enhancement, subscription or manufacturer ownership of implants, trust, and “bad actors” poisoning the market.

How firmly: He admits it is “a subjective process, and one that needs to be treated with some caution”, but says it reveals blind spots.

Companies and leaders: Written as constructive advice. Success depends on innovators “like Musk” looking “beyond technical performance and conventional risks”.

Note for mapping: This is a 2019 appearance of “orphan risks” as an established, enterprise-facing tool with a fixed list of 18, linked to an earlier OneZero/Medium post on orphan risks for tech startups and funders. The concept predates his 2026 work by years. In 2019 its orientation is toward value, enterprises and investors.

Quotes: - “We call these ‘orphan risks’ as they are often ignored, yet are frequently pivotal to an enterprise’s success or failure.” - “Rather than critique brain-machine interface technology on ethical grounds, we asked what the orphan risks landscape might look like” - “the risk map helps identify risks that cluster and converge in ways that increase the chances of truly blindsiding impacts”

2019-11-19 · the-trouble-with-connectedness-as-a-force-for-good-ab15ea149724#

“The Trouble with Connectedness as a Force for Good”

Provenance: Co-authored with Bas Boorsma (Thunderbird; A New Digital Deal), with a joint byline. The prose can’t be split between them. The Battlestar Galactica framing is characteristic of Maynard. The smart-grid, telemedicine and cities material fits Boorsma’s field. Use it as supporting evidence only.

The argument: It critiques “Net Optimism”, the belief that networked organisations are “by definition more robust”. Connectedness both builds and undermines resilience. The same networks enable surveillance (the Arab Spring, China’s “police state”, predictive policing in the US) and cascading failure (grid viruses, just-in-time food supply). They also create deep dependence: younger generations “After Google” may lack the knowledge to live without networks. The authors offer an analogy from epidemic response (Ebola) and immune systems: alternate between connection and isolation. The conclusion is to design “adaptive systems that can work equally well in isolation as they do when connected”.

Concepts: Connectedness/isolation balance. Resilience (“bend flexibly like grass or burst like bamboo”). Dependence as a form of fragility. These echo the tight-coupling argument of 2019-08-13.

Quotes (joint text): - “it behooves us to question the wisdom of uncritical and unthinking net optimism” - “connectedness, for all its promise and power, has left humanity more dependent than ever on systems that may be more fragile than we care to admit”

2020-01-01 · the-science-of-predicting-bad-behavior-2e095e9b3bcc#

“The ‘Science’ of Predicting Bad Behavior”

Provenance: His own prose, from chapter 4 of Films from the Future (2018; Minority Report), with a new opening anecdote about the Veris Prime “Trust Index” test (he scored 19; a colleague scored 2). Quotations from the studies’ authors are theirs.

The argument: The wish to predict and pre-empt “bad” behaviour is old and seductive, and repeatedly dressed as science. He builds a lineage: - phrenology (which he dates, inaccurately, to the “seventeenth century”) - Lombroso - eugenics and its role in the Holocaust - a 2011 mugshot study - fMRI studies of criminal intent and of sunscreen use - the 2016 machine-learning study claiming to classify criminals from headshots

He makes four linked claims: - Bad training sets and bias. Veris trained on white-collar felons and flags academics. - Norms are not morals. Laws encode “normative expectations”, which have criminalised homosexuality, for example. - Pre-emption. Pre-emptive action (“pre-justice”) erodes the presumption of innocence and denies people “agency over their destiny”. - Complexity. Chaos and complexity mean behaviour can be bounded but never predicted with certainty. He links this to Jurassic Park and Mandelbrot.

He argues that science is self-correcting but slow (“sometimes decades or centuries”). Until then it is “deeply susceptible to human foibles”. This is especially so in noisy, small-sample, poorly reproducible fields.

AI (important for mapping): - Algorithmic bias. “an artificial judge and jury that relies only on what you look like will reflect the prejudices of its human instructors”. - Opacity and loss of understanding. AI takes us into “big data and autonomous machines” where “we have not only trained computers to do our thinking for us, but we no longer know how they’re thinking”. He calls AI “artificial brains that we are increasingly ignorant of the inner workings of”. - The irony he closes on. In trying to predict human behaviour we may create “machines that exhibit equally undesirable behavior, precisely because they are unpredictable.”

He treats AI as a technology that amplifies old pseudoscientific temptations, and as a new kind of opaque, unpredictable decision-maker. The framing is ethical and epistemic, not catastrophic.

Cognition and being human: Neuroscience makes it feel as if “our sense of self, or our ‘soul’” may be “merely an illusion of our biology”. He asks whether neuroscience is “racing ahead of our ability to cope with what it reveals”. Free will versus determinism.

Criticises: Veris Benchmark’s marketing, clickbait science journalism, the authors of the 2016 face-classification paper, and the myth that science is value-free.

Quotes: - “there’s a danger of being caught up in the misapprehension that the scientific method is pure and unbiased” - “we have not only trained computers to do our thinking for us, but we no longer know how they’re thinking” - “we may end up creating machines that exhibit equally undesirable behavior, precisely because they are unpredictable”

2020-07-30 · life-on-mars-astrobiology-and-thinking-differently-about-risk-4f5ab6a0cca9#

“Life on Mars, Astrobiology, and Thinking Differently about Risk”

Provenance: His own prose, based on his 2018 article in Astrobiology (doi 10.1089/ast.2017.1774). The UNESCO COMEST precaution text and the Stilgoe et al. dimensions are quoted from others.

The argument: His work sits in “the twilight zone between amazing discoveries, new technologies, and the challenges of pursuing them responsibly”. He puts gene editing, “the near-unimaginable potential of artificial intelligence” and the discovery of extraterrestrial life in the same class. All are “inherently uncertain, potentially transformative, and deeply impacted by what people think, feel, and believe is right and wrong”.

Conventional risk analysis (probability of adverse outcomes, kept to acceptably low levels) “runs out of steam fast”, because “there’s more to risk than probabilities”. The type and size of harm matter, and so does who bears it: “It’s easy to make risk decisions when you’re not the one who has to suffer the consequences”. What we cannot bear to lose is often unquantifiable: loss of opportunity, loss of hope, loss of agency.

He sets out three complementary frameworks alongside risk innovation: - The IRGC risk governance framework. Evidence-based, with broad stakeholder and public engagement. - The precautionary principle. “Often misunderstood or misinterpreted” and “at the other end of the spectrum in terms of rigor”. He endorses the UNESCO COMEST (2005) formulation: morally unacceptable harm that is scientifically plausible but uncertain, a proportionate response, and a participatory process. “Precautionary principle politics aside”, it offers “a sound philosophy” for complex, uncertain and potentially catastrophic risks “before it’s too late”. - Responsible innovation. Stilgoe, Owen and Macnaghten’s four dimensions: anticipation, reflexivity, inclusion, responsiveness.

What these share is recognising “the importance of society in risk-decisions” and including people “with their fears, irrationalities, aspirations, and needs”.

Concepts and definitions: Risk innovation “seeks to transform creative ideas around how we think about risk into frameworks and processes that people want to use”. It “protect[s] what we consider to be of value”. Value runs from health, environment and economy to “scientific discovery and integrity” and to “respect, dignity, hope, belief, justice, and security”. Failure modes can depend “more on how stakeholders … feel and act than how technical systems behave”.

Past technologies: Nanotechnology and genetic engineering, from his own career. He uses them structurally: “as we know from experience”, good intentions plus “technological tinkering can lead to devastating consequences” without a broader understanding of their societal impacts.

Governance and who decides: Participatory and inclusive. Affected people and their values must shape risk decisions.

Quotes: - “But there’s more to risk than probabilities.” - “It’s easy to make risk decisions when you’re not the one who has to suffer the consequences.” - “Precautionary principle politics aside, these aspirations provide a sound philosophy for addressing complex, uncertain, and potentially catastrophic risks before it’s too late.”

Low / none posts#

None. All eight posts are medium or high.