B04 perspective notes: 2019-07-23 to 2020-07-30 (8 posts)#

I read all eight posts in full, looking at how he thinks and works as well as what he says. I read the earlier concept notes (notes/B04.md) only afterwards, as a cross-check.

Provenance and weighting - His own prose, written for the web: - 2019-07-23 neuralinks-technology-is-impressive-is-it-ethical-812afb38b19e - 2019-09-04 how-to-ensure-our-digital-legacy-isnt-lost-to-the-future-f6a226bc6792 - 2019-11-01 how-to-build-a-better-brain-machine-interface-while-not-falling-at-the-first-hurdle-cc238836a2b7. His own post about the JMIR commentary he wrote with Marissa Scragg. - His own prose adapted from his books and papers: - 2019-10-10 the-science-and-ethics-of-cloning-abeb41f1e5ad: FFTF ch. 3, pp. 46–62. - 2020-01-01 the-science-of-predicting-bad-behavior-2e095e9b3bcc: FFTF ch. 4, pp. 63–84. The Veris “Trust Index” anecdote is at FFTF pp. 63–64. - 2020-07-30 life-on-mars-astrobiology-and-thinking-differently-about-risk-4f5ab6a0cca9: his 2018 Astrobiology article. - 2019-08-13 responsible-innovation: Maynard & Garbee 2019, International Handbook on Responsible Innovation, ch. 32. Per the project rule I treat it as fully his thinking. It is in his first person throughout (his Michigan course, “As I wrote in 2015”). - 2019-11-19 the-trouble-with-connectedness-as-a-force-for-good-ab15ea149724: joint byline with Bas Boorsma. I don’t use it as evidence of his thinking. Where it only echoes a pattern established elsewhere, it is marked [co-authored]. - No Modem Futura posts and no AI-generated text in this batch. Words he quotes from others are not his views. These include Sarewitz, the 1997 cloning Declaration, UNESCO COMEST, Stilgoe et al. and the authors of the studies he discusses.

Context: This is the Risk Innovation Lab/Nexus period at ASU, just after Films from the Future came out. Most of the batch translates his books and papers into public posts, each timed to a news or film moment: the Neuralink launch, Musk’s JMIR paper, Gemini Man, the Perseverance launch.

1. How he thinks here#

How he opens. He almost never opens with a thesis. He opens with one of these: - An act of imagination. “Imagine being able to walk into a strip-mall and have thousands of microscopically-fine electrodes inserted into your brain” (2019-07-23). - A strange, delightful puzzle. “How do you write down instructions that could be read and interpreted by people living 10,000 years from now?” (2019-09-04). This is the WIPP nuclear-waste marker project. - A question to the people in front of him. “I asked students what they truly wanted to achieve” (2019-08-13). - The pull of a human desire. “There’s something quite enticing about the idea of predicting how people will behave” (2020-01-01). He starts with why an idea seduces before showing why it is dangerous. - A statement of where he stands. “I’m not an astrobiologist. But…” (2020-07-30). - A film release. Gemini Man (2019-10-10).

The moves he makes.

• He tries it on himself. Veris Prime markets a “Minority Report-like” trustworthiness test. His response: “Naturally, I took the test. I got a Trust Index of nineteen” (2020-01-01; FFTF p. 64). Then he gets colleagues to take it. One scores two, and he concludes that “academics are some of the most felonious people around”.
• The joke makes the serious point before any theory arrives. A model trained on white-collar felons flags traits “that characterize curious, independent, and personally-motivated academics”.
• The same instinct shows in 2019-09-04. An old CD-ROM of his 1990s research files turns up in the mail. He “managed to lash something together and was intrigued to dive into my professional life from 20-plus years ago”. So bit rot is tested on his own archive, not just asserted.
• He reads the study, not the headline. In 2020-01-01 he walks readers through the payoff structure of an fMRI “criminal intent” experiment in detail, with evident enjoyment (“jackpot!”). He shows it measured knowing versus reckless choices under play money. He then quotes the authors’ own caution to deflate the “Brain scans can spot criminals” headline. A 2011 mugshot study is “so small, and so narrowly defined, that it’s hard to draw any clear conclusions from it”.
• He argues by analogy of structure, not surface.
• WIPP’s 10,000-year markers and your email archive are the same problem: staying readable across time (2019-09-04).
• Innovation maps onto risk innovation: “Just as innovation is ultimately about creating value through the practical application of creativity … risk innovation focuses on the creation of value through creative approaches to potential dangers and pitfalls” (2019-11-01). He repeats the analogy in 2020-07-30.
• Tight coupling is a bell with its muffle removed: “the slightest clang of the hammer against the bell of innovation reverberates loudly round the globe at the speed of light” (2019-08-13).
• Gene editing, AI and the search for extraterrestrial life belong together because all are “inherently uncertain, potentially transformative, and deeply impacted by what people think, feel, and believe is right and wrong” (2020-07-30).
• Mandelbrot’s fractal gives the shape of behaviour prediction: we can “draw boundaries around more or less likely behaviors”, but never predict with certainty (2020-01-01; FFTF p. 76).
• He tests plausibility, between hype and dread.
• He rejects both “paralysis by analysis” and “the temptation to sensationalize and even fictionalize potential risks”. He wants “informed thinking about plausible issues, and how to navigate them” (2019-07-23).
• He marks it when he crosses the line himself: “Now we’re heading into speculative territory.”
• Technical seriousness comes first. Neuralink “isn’t vaporware”.
• With cloning, Raël’s scheme is “Stupendously complex (not to mention, implausible)”. Yet because such ideas are “increasingly garnering mainstream attention”, “we have to take the possibility of human reproductive cloning seriously” (2019-10-10).
• He hunts for paradox and irony, the places where a conventional frame breaks:
• US entrepreneurial culture “paradoxically, reflects the ideals of responsible innovation, yet rejects many of the manifestations of these ideals” (2019-08-13).
• A declaration against “mystical thinking” is easiest to find in a UFO prophet’s treatise (2019-10-10).
• In trying to predict human behaviour, we build “machines that exhibit equally undesirable behavior, precisely because they are unpredictable” (2020-01-01; FFTF p. 78).
• He holds tensions without collapsing them.
• Entrepreneurs’ duty to society sits against their duty to investors, employees and family. He takes seriously the argument that “going out of business unnecessarily is irresponsible behavior” (2019-08-13).
• Science is “one of the best tools we have” and self-correcting. But that correction “takes time, sometimes decades or centuries”, and until then science is “deeply susceptible to human foibles” (2020-01-01).
• He leaves the question of reproductive cloning open.
• He changes the question when the usual one isn’t getting anywhere. In July 2019 he wrote an ethics commentary on Neuralink. By November he says: “Rather than writing another commentary on the ethical challenges of cutting-edge brain tech, I set out to apply our work around risk innovation”. Also: “Rather than critique brain-machine interface technology on ethical grounds, we asked what the orphan risks landscape might look like” (2019-11-01).
• He uses stories and films as ways in, not as illustrations of doom. Examples include Stan Lee, Iain M. Banks’ “neural lace”, Minority Report, Gemini Man, and Jurassic Park’s chaos theory carried into another chapter. He reads a “lighthearted” io9 listicle and still finds that “it does begin to dig into the challenges” (2019-10-10). [co-authored: Battlestar Galactica as a structural parable of connectedness and isolation, 2019-11-19.]

Curiosity and delight come first. The responsibility questions follow from his curiosity; they don’t replace it. - The WIPP team’s “elegant series of marker concepts”. - fMRI as “quite incredible science”; researchers “quite literally, reading these primates’ minds”. - “What is particularly fascinating” about Raëlian ideas. - Imagining a neural link in gaming “so you could viscerally feel the action as well as see it”.

2. What matters to him#

• Agency and self-determination.
• Pre-emptive “justice” “takes us down a path where we assume that other people do not have agency over their destiny”. “There will always be an element of chance and choice that determines our actions” (2020-01-01).
• Who owns an implant, and “who has the right to write data to your brain” (2019-07-23).
• Loss of agency is a real loss (2020-07-30).
• Identity, the self and being remembered.
• The brain is “where the roots of our sense of self and identity lie” (2019-07-23).
• On bit rot: “who determines how we appear to future generations, or even whether we appear at all?” And we risk “becoming digital ghosts” (2019-09-04).
• Being human includes being legible to our grandchildren and keeping “a bedrock of reality” in a world of deepfakes.
• Fairness, and who bears the consequences.
• A “two-tier society” of the enhanced and the unenhanced (2019-07-23).
• “the value of expediency is not the value of net societal benefit” (2019-08-13).
• “It’s easy to make risk decisions when you’re not the one who has to suffer the consequences” (2020-07-30).
• Laws encode norms, not morals. Homosexuality was “illegal in the United Kingdom until 1967” (2020-01-01).
• Curiosity and discovery, as things worth protecting.
• In 2020-07-30 his list of losses that can’t be quantified includes a scientist “being denied the ability to fully apply your expertise to a challenge … that you are passionate about”.
• In 2020-01-01 the flawed test pathologises exactly the traits he prizes: curiosity and independence.
• People’s aspirations.
• He takes his students’ wish to “make the world a better place” seriously: “Such aspirations are easy to dismiss as the dreams of the young and naive” (2019-08-13).
• Scientists are “people trying to change the world for the better” (2020-07-30).
• Realising benefit. Responsibility is there “so everyone can realize the full benefits of the technology” (2019-07-23). Benefit is never an afterthought.
• A plural idea of value: “respect, dignity, hope, belief, justice, and security”, together with “scientific discovery and integrity” (2020-07-30).

What frustrates him. - Clickbait (“threw headline writers into a clickbait frenzy”). - Pseudoscience dressed as science. - The “misapprehension that the scientific method is pure and unbiased” (2020-01-01). - Responsible-innovation frameworks that are “too academic, too institutionalized and too out of touch” with the people they are meant for (2019-08-13). - Being “seduced into thinking” our digital record is permanent (2019-09-04). - A “willful disregard for future consequences” (2019-08-13). - Ethics talk that stalls in paralysis or dystopia (2019-07-23).

Where he speaks plainly on morals. He does not preach, but he does take moral stands when dignity is at stake. Eugenics’ path to the Holocaust is “heartbreaking”. To pre-emptive action against people “we arbitrarily label as ‘bad’”, his answer is “I sincerely hope not” (2020-01-01).

3. Risk as a way of thinking#

This batch is where the risk innovation vocabulary first appears in his own public prose. It shows how that vocabulary works for him.

• A new landscape needs a new mindset.
• Emerging technologies create “a risk landscape that is so different from what existed in the past that conventional ways of thinking about risk are simply not up to the task of navigating them” (2019-11-01).
• “conventional thinking about risk runs out of steam fast” (2020-07-30).
• 2019-08-13 gives the structural reasons why the landscape is new:
  • Tight coupling. Perrow and Taleb; the bell with its muffle off.
  • Latency. Harms show up more slowly than new innovations arrive, so later innovations amplify the effects of earlier ones.
  • Value mismatch. What the market rewards is not what society needs.
• Its conclusion: the “highly complex and interconnected risk landscape … increasingly requires innovation in how it is navigated”.
• This is a pre-AI argument that novel technologies call for a change of mindset, not an extra checklist.
• He builds on quantitative risk science rather than discarding it.
• In 2020-07-30 he states the conventional method fairly: “estimating the probability of adverse consequences through experimentation or modeling”. Then he extends it: “But there’s more to risk than probabilities”. What also counts is the type and size of harm, who bears it, and losses that can’t be quantified: opportunity, hope, agency.
• In 2019-07-23 physiological harm is the part he is “reasonably confident” conventional science and regulators can handle. He draws on “many years” of his own nanoparticle work and his “respect for the scientists and regulators”, but adds that they “have to be open to new ideas as the technology breaks new ground”.
• Risk as a threat to value.
• “In the Risk Innovation Nexus, we start with ‘value’”, and then ask what might threaten it. That includes “potential threats to value within key constituencies” (2019-11-01).
• Risk innovation is a way of producing tools “that protect what we consider to be of value” (2020-07-30).
• In 2019-08-13 the same idea appears as “worth”: products that stakeholders perceive as “eroding rather than creating or enhancing worth”.
• The value frame does two things at once. It admits what probability can’t hold (dignity, hope, discovery), and it connects those to what innovators already care about (trust, success).
• Orphan risks. These are “often ignored, yet are frequently pivotal”. They tend to be hard-to-quantify social risks: ethical missteps, privacy, autonomy, social justice (2019-11-01). The map’s most surprising finding is the kind the frame exists to surface. The densest cluster sits in “organizations and systems”, so the greatest threats “may lie within the company itself”.
• Navigating, not managing. In risk contexts across this batch, “navigate” appears about a dozen times. “Manage” is never used of risk. Landscapes are mapped, re-mapped “as the landscape shifts and changes”, and crossed with agility and responsiveness (2019-11-01, 2019-08-13, 2019-07-23).
• Tool or mental model? Both, and the order matters.
• 2019-11-01 is the most tool-like post: a list of 18 orphan risks and a qualitative map. But he hedges it: “a subjective process, and one that needs to be treated with some caution”. Its purpose is to reveal “pitfalls and blindsides that would otherwise be missed”. The closing ask is a capacity, not a procedure: innovators “capable of looking beyond technical performance and conventional risks, and cultivating a broader understanding of the risk landscape”.
• By 2020-07-30 risk innovation is explicitly “a way of thinking about risk that generates new knowledge, understanding, or capabilities”. The IRGC framework, the precautionary principle and responsible innovation are “different approaches to applying the concepts that underlie risk innovation”. In other words, risk innovation is the umbrella mindset and the established frameworks are instruments within it.
• The defining analogy builds creativity into risk thinking: “creative approaches to potential dangers and pitfalls”. So does its user-centred aim: “frameworks and processes that people want to use”.
• Precaution without tribalism. “Precautionary principle politics aside”, he endorses the COMEST formulation as “a sound philosophy” for “complex, uncertain, and potentially catastrophic risks before it’s too late”. That formulation has three parts: harm that is scientifically plausible, a proportionate response and participation (2020-07-30). His own plausibility test from 2019-07-23 mirrors it.
• Humility, not false precision.
• “while the concept of cloning is pretty straightforward, biology rarely is” (2019-10-10; FFTF p. 52).
• Behaviour can be bounded but not predicted “with absolute certainty” (2020-01-01).
• Latency means people may become dependent “before the long-term impacts are fully understood” (2019-07-23).
• The risk map is subjective (2019-11-01).

4. Scholarship and public writing#

• The public post as the other half of the scholarship. Five of the eight posts carry scholarship into public:
• FFTF ch. 3 and ch. 4.
• The Astrobiology article (2018).
• The Handbook chapter (2019).
• The JMIR commentary (2019). This one was already public-facing scholarship: an invited risk and ethics analysis published alongside Musk and Neuralink’s own paper (2019-11-01).

Each is timed to a cultural moment, so the research reaches readers when they are already paying attention. - Teaching as a source of insight. The core puzzle of the responsible-innovation chapter came from his Michigan classroom. He found the students’ disconnect “both surprising and unsettling”, and the “mutual worth” framework “was developed” in that course (2019-08-13). - How he treats evidence. - He is exact about what a study can and cannot show: sample size, how participants were chosen, whether results can be reproduced (2020-01-01). - He places bias not in data collection as such but in “how we decide what data to collect, and how we end up interpreting and using it”. - He credits other people’s concepts openly: Cerf’s “bit rot”, Perrow, Taleb, Stilgoe et al., Sarewitz, COMEST. - Expertise does not own the answer. - He quotes Sarewitz approvingly: leaving gene-editing decisions to experts is “wrong-headed, futile and self-defeating” (2019-08-13). - Risk decisions must include people “together with their fears, irrationalities, aspirations, and needs” (2020-07-30). “Irrationalities” is included deliberately. - Transdisciplinarity. Eight posts range across: - nanotoxicology, neuroscience and fMRI; - the history of criminology and eugenics; - archival science and semiotics (WIPP), cloning biology; - venture culture, open-source governance (Debian), university design (Crow’s ASU); - astrobiology, complexity theory and science fiction.

He names his own territory: “the twilight zone between amazing discoveries, new technologies, and the challenges of pursuing them responsibly” (2020-07-30). - Accessibility. - He explains cloning technique (somatic cell nuclear transfer) in plain words. - He talks to the reader directly and with humour (“you’ll have to trust me on this”). - He gives practical advice readers can use in their own lives: open formats, being intentional about storage, archival paper (2019-09-04).

5. His role as he sees it#

• A boundary worker, not the owner of a field. “I’m not an astrobiologist. But I do work in the twilight zone…” (2020-07-30).
• A constructive partner to innovators.
• Admiration first: “deeply impressive”.
• Fairness: “to be fair, Musk has made it clear that he’s against the idea of funding neural implants with neural adverts”.
• Hope: “The good news is that there’s still time”.
• Practical help: the Risk Innovation Accelerator (2019-07-23). A shared goal of a product “good for society as well as the company’s bottom line” (2019-11-01).
• He speaks their language. He recasts responsible innovation’s four dimensions as customer discovery, pivoting and failing fast. But he doesn’t pretend the translation is enough: “In their native form, they do not necessarily lead to responsible innovation, merely to successful innovation” (2019-08-13).
• A designer of culture.
• He prefers responsibility grown from a community’s own values to rules imposed from outside. His examples are Debian’s social contract and Crow’s ASU, which shows “seemingly unchangeable cultures can evolve”.
• He also holds that “this ethos alone” isn’t enough; “external frameworks and influences” are needed too (2019-08-13).
• This foreshadows his later institution-building.
• A teacher who learns from his students. He starts from what they want, and he treats their resistance as data.
• With readers. He trusts them with nuance and with unresolved questions. He even recommends reading Raël’s Yes to Human Cloning “with a critical mind and a good dose of skepticism”, noting that “Raël is a clear and engaging writer” (2019-10-10; FFTF p. 55).
• What he refuses to do.
• Dystopian speculation: “rarely helpful”.
• Paralysis by analysis.
• Dismissing idealists as naive.
• Dismissing fringe thinkers without reading them.
• Overclaiming. His bit-rot worst cases come with “None of these scenarios are likely”. His lost files contained “probably a lot of rubbish”, and he hasn’t “felt any pangs of regret”.
• Partisan framing of precaution.
• Lecturing practitioners from the top down.
• Changing course. The move from July’s Neuralink ethics commentary to November’s risk map is a judgement about method. He decided that conventional ethical critique had little purchase on the people building the technology. So he reframed the question, still aiming at ethics and equity, in terms of value and the builders’ own success.

6. What is distinctive#

1. He experiments on himself, playfully, as a form of argument. Trust Index 19, a colleague on 2, the CD-ROM archive. Humour and self-deprecation carry the evidence.
2. He asks about responsibility from inside the innovator’s aspirations. Responsibility becomes a condition of success rather than a brake, without dropping the marginalised, equity or the communities affected.
3. He is disciplined about plausibility. Neither hype nor dread. He marks speculative territory explicitly, and takes implausible things seriously once the ideas start gaining traction.
4. Risk is defined around value, not probability alone. Hope, agency and scientific discovery count as things that can be lost. Few risk scholars list “loss of hope” or a scientist’s thwarted passion beside harm to health.
5. Risk innovation works as an umbrella mindset. Precaution, the IRGC framework and responsible innovation become instruments within it. That lets him use several frameworks without tribal allegiance to any.
6. A long historical lineage, argued from close reading. He places AI face-classification in the line of phrenology, Lombroso and eugenics, but argues it from the details of the studies, not by association. He sees AI’s newest problem as an irony: “artificial brains that we are increasingly ignorant of the inner workings of”, and prediction that produces unpredictability. In a 2018 book and a 2020 post, this anticipates today’s opacity concerns but frames them through agency and bias.
7. Structural reasons for a new mindset. Tight coupling, harms that surface more slowly than new innovations arrive, and value mismatch. Articulated before generative AI, these map directly onto AI’s speed and reach.
8. Generosity toward unusual thinkers, entrepreneurs, regulators and people he disagrees with.
9. Long-horizon imagination aimed at the everyday. Ten-thousand-year warning markers turned onto your email and your grandchildren.

7. Posts in this batch that best reveal how he thinks#

1. 2020-01-01 the-science-of-predicting-bad-behavior-2e095e9b3bcc. The whole method in one piece: the pull of the idea, a self-experiment, close reading, history, norms versus morals, complexity and humility, then the AI irony.
2. 2019-11-01 how-to-build-a-better-brain-machine-interface-while-not-falling-at-the-first-hurdle-cc238836a2b7, read with 2019-07-23 neuralinks-technology-is-impressive-is-it-ethical-812afb38b19e. The visible pivot from ethics commentary to risk innovation: value first, orphan risks, the landscape, and his hedge on subjectivity.
3. 2020-07-30 life-on-mars-astrobiology-and-thinking-differently-about-risk-4f5ab6a0cca9. His own description of his role, and his clearest statement that risk innovation is a way of thinking that builds on probability and goes beyond it. Includes the losses of hope and agency, and the plurality of frameworks.
4. 2019-08-13 responsible-innovation. Learning from students; meeting a culture on its own terms; tight coupling, latency and value mismatch; designing culture rather than imposing rules.
5. 2019-09-04 how-to-ensure-our-digital-legacy-isnt-lost-to-the-future-f6a226bc6792. A lateral connection driven by curiosity, personal tinkering, a measured tone, and the human stakes of memory and identity.

Close behind: 2019-10-10 the-science-and-ethics-of-cloning-abeb41f1e5ad, for his generosity to Raël and for leaving the ethical question open.