S4. Risk innovation materials: what they add to the map#
Supplementary reading for 05-maynard-risk-and-ai-map.md. Share S4: the ASU Risk Innovation Nexus materials (2019–2020), the Risk Innovation Planner files (template and OpenAI hypothetical, posted 2023), two Nexus blog posts (2020), the NSF TIP RFI comments (2023), the counter-influence operations research guide (2022), the ATP-Bio JLME paper (2024), the TechTrends interview (2023) and the Slate ChatGPT-course essay (2023). Every item was read in full. Prepared 26 September 2026. This file records Andrew Maynard’s thinking only; it makes no comparison with any other material.
0. How to use this file#
Citation conventions.
- Files are cited by path under Resources/maynard-papers/.
- PDF items are cited by PDF page (“p.”) unless noted.
- The JLME paper is cited by journal page (pp.553–569 = PDF pp.1–17).
- The counter-influence guide’s printed page numbers equal its PDF pages.
- Markdown items have no pages and are cited by section.
- Quotations are exact, including the source’s typography. Where PDF extraction ran words together, the quote was taken from a clean passage or the point is paraphrased.
- “My reading” marks interpretation.
- Map references use the map’s own labels: C1–C17 for commitments, §5.x for concept tables, T1–T11 for threads, and “tension n” for §8 items.
Provenance tiers used below. Only his own writing counts as evidence of his thinking, so each item is weighted by tier.
| Tier | What it is | Items | Weight |
|---|---|---|---|
| A | His signed prose | 2020 Coronavirus post; 2020 “A New Chapter”; the Director’s Note in the Culminating Report (PDF p.2); NSF TIP RFI comments (2023); Slate (2023) | Full |
| B | His spoken words, quoted by interviewers | TechTrends (2023): direct quotes only; the surrounding paraphrase is the interviewers’ | High for quotes; low for paraphrase |
| C | Multi-author work on which he is first author | JLME ATP-Bio paper (5 authors, 2024); counter-influence (CIO) guide (6 authors, 2022) | Shared positions. In the CIO guide, ch.4 (AI ethics) reads as MIT Lincoln Laboratory’s (“our experience developing human-in-the-loop AI/ML systems”, p.27); chs.3 and 5–7 carry his risk-innovation framework and cite his work, but individual authorship is unknown |
| D | Unsigned output of a programme he directed | Nexus website pages, Introduction sheet, definition, scenario and question cards, infographic, five case studies, Planner template, body of the Culminating Report | Programme positions under his direction, not his prose. The case studies were probably drafted by staff or students; the Report credits a programme manager (Marissa Scragg), a stakeholder-engagement coordinator (Jessica Givens) and graduate students (PDF pp.15–16) |
| E | AI-generated | The filled-in entries of the Risk Innovation Planner “OpenAI hypothetical” | Excluded as evidence of his thinking (see item 12) |
Proportion. Orphan risks dominate the surface of these materials, because the Nexus was built to deliver them as a toolkit. The materials themselves place orphan risks inside a larger apparatus, summarised in the Culminating Report’s formula “VALUE + STAKEHOLDERS + ORPHAN RISKS + TOOLS = RISK INNOVATION MINDSET” (PDF p.6). The durable ideas here are threat to value, reciprocity with stakeholders, complementarity with conventional risk methods, and a “mindset”. The eighteen named orphan risks are the operational checklist that sits on top of those ideas. The entries below are weighted accordingly.
1. Risk Innovation Nexus: Introduction sheet (2019)#
File: web/2019_Risk-Innovation-Nexus_Introduction.pdf (1 p.)
Provenance: Tier D. An unsigned one-page programme overview, © 2019 ASU.
The argument. - Radical technologies, new ideas and shifting social norms are reshaping the risk landscape, so organisations “need to be equally innovative in how they think about and act on risk”. - Focusing on areas of value to “enterprises, investors, customers, and communities” lets them spot “orphan risks” before those risks blindside them. - The Nexus rests on three pillars: education and training; insights and analysis; resources and tools.
Key concepts: risk as a threat to value; orphan risks; “socially responsive innovation”; the risk innovation mindset.
What this adds to the map. - New framing: responsibility as a competitive advantage. The Nexus aims to let organisations “succeed because of how they approach responsible and ethical innovation, rather than in spite of it” (p.1; repeated in the Culminating Report, PDF p.4). The map describes risk innovation as “support for progress rather than a brake” (§5.1). That is right, but it misses how he addressed innovators: in terms of their own success. My reading: this is a deliberate engagement strategy of meeting industry in its own vocabulary, and it bears directly on how he would address technology leaders. - Deeper evidence for C3: the programme’s founding premise is parallel innovation in risk thinking.
2. Risk Innovation Nexus: orphan risks and core concepts (website pages, 2019–2020)#
File: web/2019_Risk-Innovation-Nexus_Orphan-Risks-and-Core-Concepts.md (twelve riskinnovation.org pages combined)
Provenance: Tier D. Unsigned site pages from the programme he directed. The “Further Information” list points to his 2015 Nature Nanotechnology column, his January 2016 Conversation piece and a December 2018 Medium post.
The argument. - Why conventional methods fall short. For innovators, “established rules of risk identification, assessment, and management alone simply do not work” (Risk Innovation section). Technological, social and organisational risks intertwine into a landscape that conventional tools cannot navigate. Risk innovation offers “novel approaches to risk that complement existing ones” (same section). - Threat to value. The conventional definition of risk, the probability of harm, “runs out of steam rapidly when an enterprise faces risks that are not easily quantifiable, or are not associated with clear causative links”. Worse, “too often, risks that aren’t addressable within this framework are simply pushed to one side, or overlooked” (Threat to Value section). Value includes “aspirational value—value that has yet to be achieved, but is nevertheless important”. - Why risks become orphans. The key passage explains orphaning by incentive, not ignorance. “The conventional risks that organizations tend to plan for are those where there is a clear and direct return on investment.” Other risks are dropped “because it’s harder to make the connection between investing in risk mitigation and short-term profits, and not necessarily because the risks are not recognized” (Orphan Risks section). - The taxonomy. Eighteen orphan risks sit in three domains: - organisations and systems; - social and ethical factors; - unintended consequences of emerging technologies.
They are described as “not inclusive of all risks”. - The AI Optimize scenario is summarised; see item 4. - The five case studies are indexed; see item 7.
Key concepts: threat to value, including aspirational value; orphan risks as risks that are recognised but unfunded; the risk landscape (to be navigated, even “exploiting the risks to create opportunities for success”, Risk Landscape section); the Risk Innovation Mindset (“not only value for the enterprise, but value for those that are touched by it”, Mindset section).
What this adds to the map. - Earlier origin of a 2026 idea. The map’s orphan-risks row (§5.1) gives three stages: 2018 “too ill-defined, too complex, or too irrelevant”; 2019–21 a mapping method; 2026 “known but unowned, with no tools or accountability” [mixed]. But “known but unowned” is already here in 2019: the risks are orphaned “not necessarily because the risks are not recognized” but because mitigation cannot be tied to short-term returns. The Planner template (item 11) adds “no agreed upon tools, standards, or mitigations”. So this part of the 2026 [mixed] formulation is securely his programme’s, not Fable’s. My reading: the ROI logic also prefigures at least the “measure” and “affordability” filters of the 2026 paper’s “four filters”. The frontier-AI application of those filters remains the part whose authorship is uncertain. - Deeper evidence and a qualification (Andrew’s note 1). The site keeps conventional assessment (“alone simply do not work”; “complement existing ones”). It also names the mechanism behind Andrew’s note 2: a framework built on quantification and clear causation does not merely miss what it cannot measure; it actively sets it aside. This is the 2019 form of the argument that methods and numbers can give false comfort. - AI in 2019. Both the Loss of Agency dimension and the AI Optimize scenario appear in the core pages; see items 3 and 4.
3. Risk Definition Cards (2019)#
File: web/2019_Risk-Innovation-Nexus_Risk-Definition-Cards.pdf (36 pp.: one domain card and one definition card for each of the 18 orphan risks)
Provenance: Tier D. Unsigned, © 2019 ASU.
The argument. Each orphan risk gets a short working definition. Four matter most for the map. - Social and ethical factors is defined through behaviour and cognition. Risks are “deeply affected by how people behave, including how perceptions, cognitive biases, and mental shortcuts (heuristics) affect the decisions they make” (p.1). - Unintended consequences of emerging technologies are risks that “often fall within conventional risks domains like health and the environment, but are sufficiently novel that they elude established risk assessment and management approaches” (p.13). - Loss of Agency. Technologies that reduce people’s ability to make decisions “lead to a lack of accountability and traceability in consequences arising from decision-making”. The card adds that this is “particularly relevant to the use of algorithms based on technologies such as machine learning, where there is a lack of understanding around how and why decisions are made” (p.24). - Black Swan Events. “Some black swan events are unpredictable and/or unmanageable.” Others can be prepared for “by developing resilient business practices” (p.22).
Other cards: - Perception, “irrespective of whether their perceptions are based on evidence or logic” (p.2); - Privacy, where “attitudes toward data use can turn on a knife-edge” (p.8); - Co-opted Tech, including products “co-opted for military purposes” (p.16); - Intergenerational Impacts, citing germline editing (p.18); - Bad Actors, whose conduct can “muddy the waters for other businesses” (p.26); - Reputation & Trust, where “the risks of actions that signal a lack of trustworthiness are often overlooked” (p.30).
Key concepts: loss of agency, meaning accountability and traceability under opaque machine learning; black swans handled through resilience; perception as a risk in its own right, whatever its evidential basis; bad actors as an industry-wide reputational commons.
What this adds to the map. - Earlier AI content. The map says AI “arrived late” and was “one strand of converging technologies” until 2021 (§2; §5.6). The 2019 Loss of Agency card already names opaque machine-learning decisions as a source of lost agency and lost accountability. The map’s “drain of human agency” row (§5.7) starts with the 2018 dependency risk and jumps to 2024; this card belongs between them. - Qualification to tension 2 (plausibility against tails). Low-probability, high-impact events were a standard category in his applied toolkit from 2019. They were handled by preparedness and resilience, not by probability estimates. My reading: the plausibility test governs how he judges speculative scenarios in public debate, while his practical tools always kept room for the unforeseeable. The tension is narrower than the map implies. - Andrew’s note 1. The unintended-consequences card places the novelty problem inside conventional domains: health and environment remain the frame, but novel behaviour eludes established methods. Conventional assessment is extended, not discarded. - Deeper evidence for C9 (perception as signal) and C14 (the “cognitive biases, and mental shortcuts” language appears in his applied tools a year after FFTF).
4. Risk Scenario Cards and the “AI Optimize” scenario (2019)#
Files: web/2019_Risk-Innovation-Nexus_Risk-Scenario-Cards.pdf (16 pp.; seven hypothetical scenarios); web/2019_Risk-Innovation-Nexus_Scenario_AI-Optimize.pdf (2 pp.; the AI scenario as a stand-alone sheet, identical in text)
Provenance: Tier D. Unsigned, © 2019 ASU.
The argument. Seven teaching scenarios. Each is paired with five questions for surfacing orphan risks and a map of the “critical risk dimensions” involved. - Powered Brain: transcranial stimulation for concentration. - AI Optimize: machine learning that automates business decisions. - Fossil-Be-Gone: biofuels from genetically modified bacteria. - Adapt-a-Seed: gene-edited climate crops. - Seven Twenty Four: cashless, RFID-tracked stores. - Bright Futures: CRISPR embryo editing for advantage. - Trust Me: an AI “trustworthiness index” for people entering your home. - No-Pain Meat: bioprinted meat.
The recurring device is a sincere founder. “You are certain that your use of this technology can benefit users” (Powered Brain, p.1). “You are convinced that your technology is safe, although little research has been carried out on the long term safety of your products” (Adapt-a-Seed, p.7). “you firmly believe that this is a technology that will improve lives if developed and used responsibly” (Trust Me, p.13).
AI Optimize (Scenario Cards pp.3–4; stand-alone sheet pp.1–2): - A deep-learning platform needs “full access” to company data, including employee monitoring. - “In most cases, it is not possible to trace how AI Optimize’s algorithms arrived at their recommendations.” - A premium tier makes key decisions “without any human intervention”, with quarterly profits exceeding projections by “60%-90%”. - Its key questions cover privacy, “loss of agency in terms of accountability for decisions and actions”, organisational culture, trust, and “the evolving global politics around the use of data and AI”. - Its critical dimensions include Black Swan Events, Co-opted Tech, Loss of Agency, Geopolitics and Governance & Regulation.
Key concepts: the sincere founder as the unit of analysis; the tension between commercial reward and delegated, opaque decisions; the geopolitics of AI and data (2019).
What this adds to the map. - Myopic benevolence operationalised. The book’s diagnosis (C10; §5.2 “Myopically benevolent science”) became a training device a year after publication. The user is cast as a founder who is certain, and the exercise is to see past that certainty. This is deeper evidence that C10 was a working method, not only a literary theme. - An early “economic gradient” (my reading). AI Optimize shows a commercial reward that pulls toward handing decisions to an opaque system: 5–10% gains with humans in the loop, 60–90% without. That is structurally the “economic gradient” the map dates to 2024-07-13 (§5.8), here applied to decision authority rather than manipulation. The map’s structural-incentives row (§5.2) could cite it, marked as unsigned programme material. - AI as an object of applied risk tools in 2019. Automation of decisions, opacity, accountability, workplace surveillance and AI geopolitics all appear here, with Trust Me as a Predictim analogue. This supports adding a 2019 entry to T3’s “Development” line.
5. Orphan Risk Question Cards (2020)#
File: web/2020_Risk-Innovation-Nexus_Orphan-Risk-Question-Cards.pdf (23 pp.; PDF created September 2020)
Provenance: Tier D. Unsigned.
The argument. The cover states the premise: “Modern risks from radical new technologies, innovative ideas, and shifting norms need more than conventional risk management tools” (p.1). Each of about 45 questions is linked to the orphan risks it should prompt users to review. Examples: - “What are some of the ways in which your product could potentially impact mental or emotional health?” (p.20) - “In what ways might your product reduce the ability of users or other stakeholders to make informed decisions?” (p.21) - “How might the data you have privileged access to be used in ways that may compromise others?” (p.12) - “If your organization changes leadership or is acquired by an outside company, could your technology be used in ways that are harmful or dangerous” (p.19) - “What are some of the low probability but high impact future risks associated with your product?” (p.18)
Key concepts: questions as the working interface of the method; cross-linking between risks, since one question opens several.
What this adds to the map. - Andrew’s note 1. The premise is “more than”, not “instead of”, conventional tools. - Seeds of the cognitive thread in applied form. Harm to “mental or emotional health” and to users’ capacity “to make informed decisions” appear in 2020 in a product-risk checklist. These are general, not AI-specific. My reading: they are an applied precursor of the later concern with AI’s action on judgement (C14), sitting between the 2016 neurotechnology essays and the 2023 language turn. - Governance changes as a risk. The change-of-control question (p.19) is an early, general form of a concern the map finds only in governance commentary after 2023: that a mission can be lost when ownership or leadership changes.
6. Risk Landscape Infographic (2019)#
File: web/2019_Risk-Innovation-Nexus_Risk-Landscape-Infographic.pdf (1 p.)
Provenance: Tier D. Unsigned. Only the text layer could be read: the 18 orphan-risk labels in three domains under the title “RISK LANDSCAPE”. The graphic arrangement could not be inspected.
The argument and concepts: a visual summary of the taxonomy.
What this adds to the map: nothing beyond items 2–3. It confirms the taxonomy was stable across the programme’s materials.
7. Case studies: Flo, Google Project Maven, Predictim, Theranos, TOMS (2019–2020)#
Files: web/2019_Risk-Innovation-Nexus_Case-Study_Flo.pdf, …_Google-Project-Maven.pdf, …_Predictim.pdf, …_Theranos.pdf, web/2020_Risk-Innovation-Nexus_Case-Study_TOMS.pdf (2 pp. each)
Provenance: Tier D. Unsigned, © 2019 ASU, in a common template: the company, what went wrong, the consequences, lessons learned, areas of value for enterprise, investors, customers and community, a risk landscape map, and ways to navigate. They were probably drafted by programme staff or students. Treat them as programme positions, not as his prose.
The arguments. - Flo (a period-tracking app that shared health data with Facebook). The central claim: “Their enterprise wasn’t damaged because they sold personal data, but because customers had a different perception of what the company did with the data” (p.1). Flo “could have continued to collect data while managing the data in a way that addressed customers’ concerns” (p.1). The loss of agency was “perceived and actual” (p.2). Trust-undermining actions “can be devastating—even when they make short-term financial sense” (p.2). - Google Project Maven (an AI contract for military drone imagery). Employees’ worldview clashed with the contract. Leadership tried to keep it quiet; more than 4,000 staff protested and senior engineers resigned. The company had internal networks for dissent, “and there were early warnings within these of potential issues” (p.1). The lesson is to take “workforce aspirations, concerns, and worldview seriously”: either decline the contract or proceed “only … with the clear buy-in of critical stakeholders, including the workforce” (p.1, paraphrased where extraction broke the text). - Predictim (an AI that scored babysitters from their online footprint). Concerns ranged “from algorithmic bias to the growing power asymmetry between employers and employees”. “In appealing first and foremost to their client, the parent, Predictim actually removed the need for consent” (p.1). The fix: transparency “rather than delivering results based on a black box algorithm” (p.2), and attention to the job-seekers it “lost sight of”. - Theranos. A founder admired for “willingness to break the rules”, with “no boundaries placed on its top leader”. “Theranos needed whistleblowers because the company didn’t have an organizational culture that supported transparency and trust” (p.1). The remedy includes structures that “allowed for dissenting opinions to be heard” (p.2). - TOMS (one-for-one shoe giving). “When good intentions create bad outcomes” (p.1): donations harmed local producers. The questions it poses: “do we know how our actions affect the communities we’re serving?” and “have we engaged community members and experts in our planning process?” (p.2).
Key concepts: harm routed through perception, trust and reputation back to the firm; employees and whistleblowers as early-warning channels; consent owed to non-customers; good intentions producing harm; black-box opacity.
What this adds to the map. - Sharpens tension 4 (“Whose value?”). The case studies show the frame’s enterprise-facing logic plainly. Flo’s harm is recast as a perception failure, and the recommended path keeps the data practice while managing concern. Predictim’s job-seekers and TOMS’s local producers are at least named, but mainly through the “outrage of community members and their advocates” (Predictim, p.2) and reputational cost. My reading: these materials put harm to people without leverage onto the map mainly when it can feed back to the firm. This is programme material, not his prose, and it is exactly the limitation the map’s tension 4 describes. See also item 14, where the assumption is stated outright. - Partly fills a gap the map lists. The map lists “whistleblowers and employees inside firms” as little developed (§8 Gaps). In the Maven and Theranos cases, internal dissent, employee advocacy and whistleblowing are early-warning signals that a healthy organisation should hear. The phrase “early warnings” is used for employees’ internal networks in 2019. - Deeper evidence for C10 (TOMS, Theranos), C11 (Predictim’s power asymmetry and consent) and C5 (the Maven workforce as a stakeholder with a say). - AI cases in 2019. Maven and Predictim add two more AI-specific applications of his risk tools before 2021 (see item 4).
8. Risk Innovation Nexus Culminating Report (October 2020)#
File: web/2020_Risk-Innovation-Nexus_Culminating-Report.pdf (17 pp.)
Provenance: - The Director’s Note (PDF p.2) is Tier A, signed by Maynard. Its text is nearly identical to the “A New Chapter” post (item 9). - The body is Tier D. - The closing thoughts are signed by Scragg and Givens (PDF p.15). - A foreword is by Ji Mi Choi of ASU’s Entrepreneurship + Innovation institute (PDF p.3).
The argument. Entrepreneurs are “deeply risk-savvy”, yet “risk-savvy all too easily becomes a risk-liability as new and unexpected threats to value emerge” (PDF p.5). Some threats “have a familiar feel”, such as health, wealth and environment; many do not. Risk innovation brings “an entrepreneurial mindset” to the risk landscape “to protect and create value” (PDF p.5). The approach is compressed into four questions: “What is important? To whom is that important? What could threaten value? How to respond with agility?” (PDF p.6).
Outputs listed: - the Planner as “keystone tool”; - definition, scenario, question and value-identifier cards; - 15 workshops; - a Canvas module; - case studies; - a blog; - a list of Maynard publications, 2015–2019 (PDF pp.8–12).
The milestones (PDF p.13): - “[03/13] Risk Innovation Nexus director starts to develop ideas that underpin the Nexus, while teaching Master of Entrepreneur students at University of Michigan”; - Risk Innovation Lab launched August 2015; - Accelerator formed March 2017; - MVP tools including the Planner, January 2019; - a pilot with entrepreneurs “to test and iterate” tools, January–April 2019; - the rename to Nexus, September 2019; - a presentation to the National Academies Committee on Science, Technology, and Law, October 2020.
Key concepts: the four questions; value + stakeholders + orphan risks + tools = mindset; win-win innovation.
What this adds to the map. - Earlier origin. T1 says the frame was “Founded in 2016 (2016-01-11)”, and §5.1 says “from a 2015 column”. By his own account the ideas began in March 2013 with entrepreneurship students at Michigan, while his professional focus there was risk assessment. The Lab dates from 2015. The map’s Phase 0/1 boundary (§7) should note this. - The four questions are a compact, portable statement of the method that the map lacks. They are close to the map’s lens 1 (§9) but add “How to respond with agility?”, which puts action and adaptation ahead of analysis. - Proportion. The formula puts orphan risks as one of four elements, which supports the map’s treatment of orphan risks as a tool inside a wider framework. - Evaluation gap. The report lists outputs (workshops, tools, pilots) but gives no outcome data. This is consistent with the map’s §8 gap; see item 15 for his own acknowledgement.
9. “A New Chapter for the ASU Risk Innovation Nexus” (2020-10-23)#
File: web/2020_Risk-Innovation-Nexus_A-New-Chapter.md
Provenance: Tier A. His signed blog post. The corpus post 2020-11-05 risk-innovation-and-the-future reuses much of its text and adds material on complex systems.
The argument. Seed funding is ending, and the tools remain free. It restates the Michigan origins, the 2016 Idea Enterprise support and the 2017 Accelerator, and gives the definition of orphan risks as “those hard to quantify and easy to ignore risks that nevertheless have a habit of coming back to bite”.
What this adds to the map: confirms the dates in item 8 in his own words. There is little new conceptually, since the Substack version is already in the corpus.
10. “Risk Innovation in a Time of Coronavirus” (2020-03-27)#
File: web/2020_Risk-Innovation-in-a-Time-of-Coronavirus.md
Provenance: Tier A. His signed post launching the Nexus blog. Not in the corpus.
The argument. - Deference to the professionals. “risk innovation is intended to complement and enhance existing risk assessment and management approaches”. For anyone managing the immediate health impacts, “this website should not be your first port of call!”: go to the CDC, the WHO and local authorities. - The wider threats. Beyond infection, COVID-19 threatens what matters to people and organisations: stress and anxiety, mental health under isolation, “the potential socioeconomic impacts of extreme mitigation strategies”, and “the consequences of disproportionate access to life-saving healthcare”. These are “hard to quantify and easy to overlook”, and they are orphan risks. - What risk innovation is. It is “an approach to risk that considers threats to what others value, as well as what’s important to you”, and one that “dovetails into more formal ways of assessing and managing risks”. Beyond that, it is a mindset for a world where risks “simply cannot be navigated effectively with the risk management tools of the past”.
Key concepts: complementarity; the risks of mitigation itself; equity of access; resilience and foresight.
What this adds to the map. - The clearest statement in his own words for Andrew’s note 1. Faced with a real crisis, he sends people first to conventional public-health risk management and positions risk innovation as the layer that catches what that system does not. The map’s C3 (“necessary but no longer sufficient”) is correct, but it should carry this ordering explicitly: conventional methods first, risk innovation built on top. - Symmetric risk applied in real time (C6). The risks of “extreme mitigation strategies” are counted alongside the disease, a 2020 instance of the risks of precaution itself. - Justice (C11). “disproportionate access to life-saving healthcare” is named as a threat to value.
11. Risk Innovation Planner: template (© 2019; posted November 2023)#
File: web/2023_Risk-Innovation-Planner_Template.pdf (2 pp.)
Provenance: Tier D. The unsigned 2019 programme tool, re-posted on andrewmaynard.net in 2023.
The argument. The Planner addresses orphan risks, defined as “often-overlooked risks to success for which there are no agreed upon tools, standards, or mitigations already in place” (p.1). It has three steps: 1. Name three areas of value for each of enterprise, investors, customers and community. 2. Circle the orphan risks that threaten them, and describe how. 3. Choose three small actions for the next quarter, each with a what, a why and a how, and each “specific enough to complete within 2-4 weeks”. Examples: “read an article or book; talk to a customer; write a blog post” (p.2).
It closes with a “Quarterly Reflection” on which actions worked (p.2).
Key concepts: rapid, iterative, low-cost risk planning; quarterly review; community as a named stakeholder.
What this adds to the map. - Earlier origin: “no agreed upon tools, standards, or mitigations” is the 2019 form of the 2026 “no tools or accountability” (see item 2). - Method (my reading). The tool is frugal by design, to be completed “in 30 minutes” with a founder (2023-11-21 post). Its modest, iterative actions and scheduled reflection are adaptive learning at small scale. This sits comfortably with Andrew’s note 2: the tool builds a learning loop rather than a prediction.
12. Risk Innovation Planner: OpenAI hypothetical (2023-11-21)#
Files: web/2023_Risk-Innovation-Planner_OpenAI-Hypothetical.pdf (2 pp.); web/2023_Risk-Innovation-Planner_OpenAI-Hypothetical_text.md (annotation text)
Provenance correction (Tier E). The README and the header of the _text.md file present the entries as “Author: Andrew Maynard”. According to his companion post (corpus: 2023-11-21 ai-and-risk-innovation), the entries were generated by ChatGPT, playing the board of a hypothetical company closely modelled on OpenAI, from anonymised documents he uploaded. He guided the session, had ChatGPT summarise its answers, and did one thing himself: “I needed to map ChatGPT’s orphan risks to those on the planner”. So his contribution to the filled planner is the choice of exercise, the prompts, and the boxing of eight categories:
- Governance & Regulation;
- Organizational Values & Culture;
- Reputation & Trust;
- Ethics;
- Perception;
- Social Justice & Equity;
- Black Swan Events;
- Loss of Agency.
The prose in Parts 2 and 3 and the quarterly reflection are AI-generated and should be excluded as evidence of his thinking, as the map already does for GPT scenario assessments. The README and _text.md header should be amended.
What the exercise contains (for the record, not as his view). Areas of value included mission, safety and ethics, capped investor returns, and equitable access. ChatGPT’s orphan risks: - governance misunderstandings; - ethics and mission misalignment; - access disparities; - AI policy influence; - “competence vs. control”; - public perception; - “AGI development pace”.
Its three actions: a mission-alignment review of commercial work, a public forum on the pace of AGI development, and an independent governance audit.
His own framing (Tier A, from the corpus post). - He says the output was what he would expect from the method: “Not surprisingly the responses to the Planner weren’t unexpected to me — this is the bread and butter of what I do”. - He calls the steps “quite basic”. - He concludes that the Planner is “a powerful tool for thinking through challenges around AI development that previous experience in tech innovation and formal business and risk management training simply do not prepare people for”. - “What the Planner does not do is provide answers to problems.”
What this adds to the map. - Provenance. A correction to the source files, as above. - Weak, endorsed evidence (AI output he endorsed). He would have expected an OpenAI exercise to surface the governance structure, mission drift under commercial pressure, pace, and the gap between capability and control. It is endorsement of AI output, not his analysis, and should be marked that way if used. - Deeper evidence for the map’s point that the frame moved to AI developers in 2023 (T1 “Development”). Here the tool is aimed at a frontier AI company in the week of its governance crisis.
13. Response to the NSF TIP Directorate roadmap RFI (July 2023)#
File: web/2023_NSF-TIP-RFI_Maynard-Comments.pdf (4 pp.; PDF created 25 July 2023; posted on andrewmaynard.net September 2023)
Provenance: Tier A. His signed submission, as “Expert in emerging technologies and advanced technology transitions”.
The argument. - Framing. NSF’s Technology, Innovation and Partnerships (TIP) directorate should adopt “Advanced Technology Transitions” as a crosscutting theme. This would “enhance U.S. competitiveness and support workforce growth” (pp.1–2) and align with the CHIPS Act, including its section on “Ethical, legal, and societal considerations”. - The diagnosis (p.2): - “we are at a scientific and technological tipping point in human history”; - “Emerging AI foundation models alone have seen the timescale associated with social disruption move from years to months”; - the result is “potentially catastrophic economic, social, and environmental failures that are rooted in conventional thinking, naïve assumptions, siloed understanding, and limited approaches to beneficial and responsible innovation”; - these are failure modes “not only insurmountable through conventional thinking but are obscured through established – and often outmoded – approaches to technology innovation”. - What is needed. New thinking, skills, organisational structures and leaders to “steer these technology transitions toward more vibrant, promise-filled, and equitable futures” (p.2). “At present, an integrated, transdisciplinary, and use-inspired approach to advanced technology transitions is lacking” (p.3). - The proposed research domains (pp.3–4) include: - “Failure modes, best practices, and emerging principles, associated with historic advanced technology transitions”; - governance “spanning the spectrum of public engagement, soft law, agile governance, hard-law regulation”; - equity; - “Public engagement, democratic decision making”; - the arts and humanities; - “Novel theories, models, and approaches to risk”; - foresight; - misinformation; - job loss and gain; - education. - It cites Maynard & Dudley (2023) in Nature Nanotechnology on lessons from nanotechnology.
Key concepts: ATT as a research field; compressed disruption timescales; failure modes hidden by outmoded approaches; the policy register of competitiveness together with societal benefit.
What this adds to the map. - Earlier date for the timescale mismatch. The map dates the “timescale mismatch” to 2025-04-06 and the timescale inversion to 2021-04-09 (§5.4). Here, in July 2023, he says AI has moved disruption from “years to months”. Add it to §5.4. - Andrew’s note 2, in his own words. The claim that failure modes are “obscured through established – and often outmoded – approaches” goes beyond “methods are incomplete”. It says the methods themselves can hide danger. That is the core of the hubris-of-method concern, stated in 2023 about technology innovation generally. The document does not use the word “hubris”. - Andrew’s note 1. The answer is not to discard risk science but to add “Novel theories, models, and approaches to risk” and to learn from “historic advanced technology transitions”. Past learning is built on. - Register (my reading). This is the clearest item in the share where he argues to government in the language of national competitiveness and workforce growth, with equity, democratic decision-making and ethics carried inside the same programme. It shows how he pitches responsible development to power: as a condition of economic success, not a tax on it. - Deeper evidence for the map’s §5.4 ATT row (2023) and C8 (the tipping point).
14. Conducting Socially Responsible and Ethical Counter Influence Operations Research: A Practical Guide (January 2022)#
File: papers/2022_Conducting-Socially-Responsible-Ethical-Counter-Influence-Operations-Research_Guide.pdf (58 pp.)
Provenance: - Authors and origin. Tier C: Maynard, Corey, Greaves, Kozar, Kwon and Scragg. It is a collaboration of MIT Lincoln Laboratory, ASU’s Global Security Initiative and the ASU Risk Innovation Lab, funded through the Under Secretary of Defense for Research and Engineering, and approved for public release. - Chapter 4 (AI ethics, pp.24–30) is written from the Lincoln Lab engineers’ standpoint. - Chapters 3 and 5–7 carry his framework and cite Maynard 2015, 2018 and 2019 and Maynard & Scragg 2019. - Pages 11 and 48–53 (the individual, ten questions, principles, resources) are of uncertain authorship.
Treat the guide as shared positions.
The argument. - Why the guide exists. Influence operations use social media to “divide, deceive, and create unrest” (p.4). Countering them requires research that can itself cause harm and damage institutions. Platforms “whose business goal is to increase usership and online time, have little motivation to implement solutions” (p.7). - Ethics of social media research (ch.3). Five threads: - the gap between “can do” and “okay to do” (p.13); - privacy as context and dignity, drawing on Nissenbaum’s “contextual integrity” (pp.14–16); - harm in many forms, including dignity (pp.16–17). Harm to users is “what is important to them being threatened in some way – a concept that reflects thinking around “risk innovation”” (p.17); - uninformed and unwitting participants: “a duty not to exploit subjects who inadvertently place themselves in a vulnerable position” (p.20); - ethics considered at every stage of the research, with Jasanoff’s “Technologies of Humility” among the works cited (p.22; reference list p.55). - AI ethics (ch.4, Lincoln Lab). Bias taxonomies and twelve mitigation practices, including a “pre-mortem” that asks “What is the likelihood, magnitude of impact, and scale of each of these risks?” (p.28). - Risk innovation (ch.5). “Risk, in its simplest form, concerns the likelihood of adverse outcomes arising from decisions or actions” (p.31). Consequentialist ethics links ethics to risk. But “ethical” or “societal” risks are uncertain and ambiguous: “Such threats are subjective, rooted in human behavior, complex, and near-impossible to quantify. Yet they often play an outsized role in determining success or failure” (pp.31–32). The tools map stakeholders, values and orphan risks “on the assumption that threatening stakeholder value becomes a threat to principal agent value” (p.34). A “key tenet”: “everyone has both the responsibility and opportunity to understand evolving risk landscapes from their own perspective” (p.34). - Institutional risk (ch.6). Such risks “are as much predicated on perception as they are on evidence”. “It is rarely if ever sufficient to claim that you are behaving ethically and in the public interest if the prevailing public opinion suggests otherwise” (p.35). Research that aims to change behaviour rather than observe it is “likely to significantly elevate any potential risk” (p.37). Non-consensual interventions risk “perceptions that specific values are being imposed on subjects in an effort to influence the ways they think and behave” (p.38). - The applied case (ch.7). A team built an offline research platform on which “a chat-bot designed to engage with and develop a trusted relationship with participants” was to change their behaviour and thinking (p.41). The bot would disclose that it was not human, but in a conversational way, and “present itself as a member with similar values and interests” (p.42). Through risk-innovation templates and personas, “the team discovered gaps in their initial thinking” (p.42). - Closing lists. - The principles open with “Will not use unethical means to combat disinformation” (p.50). - A “plus one” commits to own and learn from inadvertent harm, because “it’s not possible to predict all the ways in which a capability could be misused in the future” (p.50). - It warns against “technical tunnel vision” (p.50). - Summary: “There is no recipe for conducting ethical counter influence operations” (p.54).
Key concepts: contextual integrity; dignity as the measure of harm; unwitting participants; institutional risk and the “court of public opinion”; social and political third rails; observation versus intervention; the principal-agent/stakeholder reciprocity assumption; own-and-learn humility.
What this adds to the map. - A partial integration rule for tension 3 (“Two definitions of risk, no integration rule”). Pages 31–32 give his framework’s bridge: probability of adverse outcomes is the base definition, and where “adverse” is defined by social norms and resists quantification, threat-to-value mapping takes over. This is not a full rule. It says nothing about evidentiary bars for cognitive harm. But it narrows the tension and fits Andrew’s note 1: the probabilistic definition remains the foundation. - The reciprocity assumption stated outright (tension 4). “on the assumption that threatening stakeholder value becomes a threat to principal agent value” (p.34) names the mechanism on which the enterprise-facing frame depends. My reading: where affected people cannot make their loss felt (unwitting subjects, the unconsulted, the powerless), the assumption fails. The guide partly compensates with duties that do not depend on feedback, such as the duty not to exploit unwitting subjects (p.20). The frame therefore carries two logics, prudential reciprocity and ethical duty, and the map could say so. - Early, hands-on engagement with benevolent AI persuasion (§5.8; tension 11). In 2021–22 he helped a team apply risk innovation to a chatbot designed to build trust and change beliefs for a public-good aim, presented as someone with “similar values and interests”. The guide treats deliberate influence as the highest-risk category and flags the imposition of values. His 2024-09-01 post cites this collaboration with Hazel Kwon in the same essay where he asks “who decides what is good for society?” and discusses Kahan’s finding that people are persuaded by “their sort of person”. My reading: his 2024 question has roots in practical work on benevolent machine persuasion. That work already asked who sets the values, if mainly through institutional risk. Tension 11 should be qualified: he did ask the question of persuasion he helped design, though not in the terms he later applied to AI companies. - Andrew’s note 2. Several things together: - “near-impossible to quantify. Yet they often play an outsized role” (pp.31–32); - the own-and-learn commitment grounded in the impossibility of predicting misuse (p.50); - “There is no recipe” (p.54); - the citation of Jasanoff’s “Technologies of Humility”.
These are humility as method. Chapter 4’s likelihood, magnitude and scale questions show that quantitative habits sit alongside, contributed by the engineering co-authors. The mix is layered, not exclusive. - A qualification to tension 5 (race logic), with a caution. On p.11 the argument “if I don’t invent it someone else … will” is called “nevertheless a valid one”, paired with “be aware, keep track, practice due diligence”. This sits awkwardly with his later criticism of race logic (2026-09-24 [mixed]). The section’s authorship is uncertain and it concerns individual researchers, not firms or nations, so it should not be weighted as his position. It is worth recording as a shared text he put his name to. - Partly fills the map’s gaps. The guide covers the individual’s duty to “at least raise awareness” and notes that “whistleblowing is increasingly common” (p.11). It also addresses the ethics of corporate-controlled research data (pp.8–9). Both touch thin areas in §8 (whistleblowers; platform harms). - Deeper evidence for could vs should (§5.2), structural incentives on platforms (§5.2, 2022), perception as risk (C9), dignity (C16), and “It’s good to talk” as a practice (researchers consulting “as many people and resources as possible”, p.20, quoting AoIR).
15. Maynard, Oye, Scragg, Tripp & Wolf, “Successfully Bridging Innovation and Application …” JLME 52 (2024): 553–569#
File: papers/2024_Risk-Innovation-ATP-Bio_JLME.pdf
Provenance: Tier C. Five authors, with Maynard first; he and Scragg led the workshops (p.557). The corpus post 2024-12-17 on biopreservation is his own summary, which the map already uses.
The argument. - The problem. Advanced biopreservation could disrupt systems built around fixed time constraints, such as organ allocation. The resulting barriers “lie outside many conventional approaches to risk assessment and management” (p.555). - Complementarity. Risk innovation “was conceived from the outset as complementary to more-established risk analysis and decision-making tools”, including Enterprise Risk Management, SWOT and ISO 31000. It is “designed to foster a risk-based mindset that was attuned to less conventional risks” (p.555). - Value. In conventional risk assessment, value means health, environment or fiscal security. It can also extend to “human flourishing, an equitable society, mental well-being, social and individual identity, autonomy, and dignity”. “These and similar dimensions are rarely quantifiable and are often ignored in formal risk assessments, and yet are frequently pivotal to the success of new endeavors” (p.556). - Method. - Three 90-minute workshops with 17 participants from 16 partner organisations, using a Planner-based protocol, 73 prompt areas of value, and a short AI case study (pp.557–559). - Aggregation is “an intentionally subjective process that is designed to avoid “paralysis by analysis”” (p.559). - Frequencies were normalised to a 1–10 scale for comparison (p.563). - Findings. - Ethics, perception, black swans, governance and regulation, and reputation and trust recur across the domains (p.565). - Black swans (possibly “a catch-all category for risks they could not foresee”) and governance appear in all three domains, suggesting ““safety net” strategies” and early engagement with regulators (p.565). - Self-limits. - The 18 orphan risks omit “the “risk” of hype and overblown claims of benefits” and the risks of political or ideological agendas (p.557). - The approach’s “primary purpose is to help enterprises (or associated individuals/groups) understand the risk landscape that arises at the intersection between enterprise and stakeholder value” (p.564). - “it was not designed to provide proof of positive impact”; “Follow-on research to better understand the extent to which adopting a Risk Innovation approach leads to positive outcomes would be valuable” (p.567).
Key concepts: complementarity with ERM, SWOT and ISO 31000; subjective but informative assessment; avoiding paralysis by analysis; safety-net strategies; flourishing as a value at risk; a multi-stakeholder extension of the Planner.
What this adds to the map. - The strongest multi-author statement for Andrew’s note 1. The approach was complementary “from the outset” to named conventional frameworks. C3, T1 and §2 should make the layering explicit. - Andrew’s note 2. The paper treats quantification as neither available nor necessary for the risks that matter most, and deliberately chooses subjectivity so that analysis does not stall action. It still uses light, clearly labelled quantification (normalised frequencies, “While qualitative”, p.563). My reading: he does not reject numbers. He rejects letting measurability decide what counts. - Correction to §8 Gaps (“no reported evaluation …”). The gap is real, but he names it himself (p.567). Re-tag it “[he says so]”. It is also the first reported multi-stakeholder use of the method, beyond the single firms the programme was built for (p.555). - Tension 4, sharpened. The “primary purpose” sentence (p.564) confirms the enterprise orientation in a peer-reviewed text he led. - A new detail: hype is named as a risk the taxonomy misses (p.557). This connects his earlier “brand-nano” critique (§6 T2) to the risk-innovation toolkit in 2024. - Flourishing as a value at risk in a formal risk-method paper (p.556). This is deeper evidence for C1: the value frame reaches flourishing in his applied work, not only in his essays.
16. Richardson, Oster, Henriksen & Mishra, “Artificial Intelligence, Responsible Innovation, and the Future of Humanity with Andrew Maynard,” TechTrends (December 2023)#
File: papers/2023_TechTrends-Interview-AI-Responsible-Innovation_author-copy.pdf (7 pp.)
Provenance: Tier B. The interviewers wrote the column. Only text in quotation marks attributed to him counts as evidence, as spoken words. The paraphrase (for example, that at Michigan “his work focused on risk assessment”, p.2) is the interviewers’.
The argument (his quoted words). - His formation, and an ASU that combines his “physicist mindset, understanding of risk, innovation around how we think differently about risk”. Physics taught him: “Yes, the rigor and the math were important”, but its essence is delight in seeing anew. “When I discover I’m wrong … I find it amazing” (pp.2–3). - Innovation. Innovation is putting ideas together into something “that other people are willing to invest in”, whether time, emotion or money (p.2). - AI in education. “The power of ChatGPT is in the conversations” (p.1). Generative AI’s generativity lies in “the computer-human collaboration” (p.3). Personalised, “augmented learning” (p.4). - Irreversibility. “we no longer have the luxury of making mistakes”; “we’re going to make mistakes that we can’t back up from” (p.4). - Who decides. “Currently, most of the conversations are driven by white guys who head up tech companies.” Citizens and civil society could lobby those companies and withhold support, “But to do that, people must be empowered. You can’t do that without helping them develop that technical literacy through education and learning” (p.5). - Not knowing. “we don’t have theories of advanced technology transitions. We don’t have a body of knowledge that helps us understand how to actually navigate this successfully” (p.5). - Communication. “most people are reasonably smart”. Effective persuasion means meeting people with respect and understanding “what is of value to them” (p.5). - A forecast. “somewhere between the next six months and three years, you’ll see a sudden outburst of new capabilities that are transcendent compared to what we have with ChatGPT” (pp.5–6). - Manipulation. A machine that manipulates but “cannot be manipulated back” is “like throwing a virus into an unprotected community” (p.6). - A tipping point “the likes of which we haven’t seen for at least 200 years, maybe even a thousand years” (p.6).
Key concepts: the layered identity of physicist, risk scientist and risk innovator; irreversibility; the absence of theory; citizens’ leverage through literacy; manipulation as an asymmetric contagion.
What this adds to the map. - Andrew’s note 1 in his own words. He describes his approach as a stack: “physicist mindset, understanding of risk, innovation around how we think differently about risk”. Rigour and mathematics remain; risk understanding remains; innovation is added. This is the best single quotation in the share for correcting the map’s tendency to present his positions as replacements. - Andrew’s note 2 in his own words. “we don’t have theories of advanced technology transitions” is a plain statement of the lack of understanding he says guides his humility. It comes from late 2023, when he was advocating for ATT as a field (compare item 13). - Deeper evidence for C14. It is a 2023 spoken restatement of the Ex Machina thesis, beyond the 2023 repost, with a vivid new image of asymmetric manipulation. - Deeper evidence for C5 and C11. A sharper 2023 formulation of who decides (“white guys who head up tech companies”). - Deeper evidence for §5.10 AI literacy. Literacy as the precondition of citizen power, marking the 2023 high point of his confidence in literacy that the map says later declines. - Deeper evidence for C8. A dated 2023 statement of irreversibility, between FFTF (2018) and the 2025 reversibility footnote. - A forecast on record. His 2023 expectation of a capability jump within six months to three years adds a data point to T3/T9 on how he reasons about trajectory before 2025’s “exponential blindness”.
17. “I Asked ChatGPT to Develop a College Class About Itself,” Slate, Future Tense (2023-07-16)#
File: web/2023_Slate_I-Asked-ChatGPT-to-Develop-a-College-Class-About-Itself.md
Provenance: Tier A for his prose. The frameworks he reports as ChatGPT’s (for example “RACCCA”) and the quoted prompts are AI-generated or co-created and are not evidence of his thinking. Not in the corpus.
The argument. - How the course was built. He built a six-week prompt-engineering course largely designed, run and assessed with ChatGPT. “the vast majority of the course I’m now teaching is designed, written, and executed by ChatGPT”. He added what the model left out, notably “the societal implications of large language models and A.I. chatbots” and responsible innovation, and he stayed the “human in the loop”. - Collaboration over replacement. ChatGPT “could do my job with me”, and “the real power of all this lies in human-ChatGPT collaborations”. - Teaching the failures. An early exercise shows “when and how ChatGPT can get things convincingly wrong”. - Being changed by the tool. Reading 2,000 student–ChatGPT conversations: “It’s almost as if ChatGPT is fine-tuning my brain to be a better instructor”. - His own profession. It “hasn’t made me fear for the future of my profession—at least not yet”.
Key concepts: human–AI co-creation; the human in the loop; AI as co-instructor.
What this adds to the map. - An early, benign version of reverse shaping. The map’s reverse formation† (“beginning to train us to think like them”, 2026-07-19) and formation (2026-09-24 [mixed]) have an unnoticed 2023 precursor in his own prose, “fine-tuning my brain”, offered then as a gain. My reading: the idea that AI reshapes its user is present from 2023. What changes by 2026 is its valence, not its presence. - Deeper evidence for tension 8 (catalyst against surrender) and tension 7 (adopter against critic). This is the high point of his 2023 enthusiasm, with a hedge (“at least not yet”). - Deeper evidence for §5.10 (catalyst, conversation over prompt) and for the “augmentation, not replacement” row (§5.6).
18. Cross-cutting findings#
18.1 Andrew’s note 1: quantitative risk assessment built on, not abandoned#
The evidence runs consistently in one direction across six years and all provenance tiers.
| Year | Statement | Source (tier) |
|---|---|---|
| 2019 | established methods “alone simply do not work”; risk innovation offers approaches “that complement existing ones” | Core Concepts, Risk Innovation section (D) |
| 2019 | novel risks “often fall within conventional risks domains like health and the environment, but are sufficiently novel that they elude established risk assessment” | Definition Cards p.13 (D) |
| 2020 | “intended to complement and enhance existing risk assessment and management approaches”; “dovetails into more formal ways of assessing and managing risks”; defer first to the CDC and WHO | Coronavirus post (A) |
| 2020 | “need more than conventional risk management tools” | Question Cards p.1 (D) |
| 2022 | “Risk, in its simplest form, concerns the likelihood of adverse outcomes” | CIO guide p.31 (C) |
| 2023 | “physicist mindset, understanding of risk, innovation around how we think differently about risk”; “the rigor and the math were important” | TechTrends pp.2–3 (B) |
| 2023 | calls for “Novel theories, models, and approaches to risk” alongside lessons from historic transitions | NSF RFI p.4 (A) |
| 2024 | “conceived from the outset as complementary” to ERM, SWOT and ISO 31000 | JLME p.555 (C) |
Implication for the map. C3, T1 and the “Engine” paragraph of §2 should say explicitly that conventional, probability-based risk assessment is his foundation, and that risk innovation is a layer added where it “runs out of steam”. The 2018 line that the frame “extends conventional thinking rather than replacing it” (already quoted in C2) should be carried into C3 and T1, with the items above as support.
18.2 Andrew’s note 2: humility, not hubris, about methods for AI#
Supporting evidence. - The mechanism of false comfort, from 2019. Frameworks built on quantification and clear causation push aside what they cannot handle (Core Concepts, Threat to Value section). - Methods can hide danger. Failure modes “obscured through established – and often outmoded – approaches” (NSF RFI p.2, 2023). - An admission of not knowing. “we don’t have theories of advanced technology transitions” (TechTrends p.5, 2023). - A deliberate choice of method. Subjectivity chosen to avoid “paralysis by analysis” (JLME p.559). - Preparation over prediction. Black swans handled through resilience and “safety net” strategies rather than prediction (Definition Cards p.22; JLME p.565). - Humility as procedure. An own-and-learn commitment because misuse cannot be fully predicted (CIO guide p.50), and the guide cites Jasanoff’s “Technologies of Humility”.
Qualifying evidence. - The word “hubris” does not appear in this share. The humility argument is present as mechanism and method, not as a named critique of the hubris of risk assessment. - Light quantification is used where it helps: normalised frequencies (JLME p.563); likelihood, magnitude and scale in the pre-mortem (CIO guide p.28, probably Lincoln Lab’s).
Implication for the map. The §8 “Missing methods” bullet (“no quantitative treatment of AI risks despite his risk-science background”) should be reframed. It is a considered position, visible since 2019, that measurability should not decide what counts, joined to practical, iterative tools for acting under deep uncertainty. It is not a missing capability.
18.3 Earlier origins the map dates later#
| Idea | Map’s date | Earlier evidence |
|---|---|---|
| Risk innovation’s origins | 2015 column / 2016 | March 2013, Michigan (Culminating Report PDF p.13; Tier A Director’s Note, PDF p.2) |
| Orphan risks as “known but unowned”, with “no tools” | 2026 [mixed] | 2019 site (“not necessarily because the risks are not recognized”); Planner template (“no agreed upon tools, standards, or mitigations”) (D) |
| Opaque ML decisions as loss of agency and accountability | 2024 (drain of agency) | 2019 Loss of Agency card; AI Optimize scenario (D) |
| Commercial gradient toward delegating to opaque AI | 2024-07-13 (economic gradient) | 2019 AI Optimize scenario (D; my reading) |
| Timescale compression by AI | 2025-04-06 | July 2023 NSF RFI: “from years to months” (A) |
| Benevolent machine persuasion; “who decides what is good” | 2024-09-01 | 2021–22 CIO chatbot case and value-imposition warning (C) |
| AI reshaping its user | 2026-07-19 | 2023 Slate: “fine-tuning my brain” (A) |
| Employees as early-warning channels; whistleblowers | listed as a gap | 2019 Maven and Theranos cases (D); CIO guide p.11 (C) |
18.4 Corrections and qualifications, by map location#
- §5.1 Orphan risks row. Move “known but unowned, with no tools” to 2019 (programme material), and keep only the frontier-AI filters and apparatus as [mixed].
- §8 tension 2. Qualify it: tails were built into his applied tools from 2019 through black swans and resilience.
- §8 tension 3. Note the partial integration in the CIO guide (p.31) and JLME (p.556). Conventional risk assessment is the special case of threat to value where value is measurable. Evidentiary bars for cognitive harm remain unaddressed.
- §8 tension 4. Add the explicit reciprocity assumption (CIO guide p.34), the “primary purpose” sentence (JLME p.564) and the case-study pattern. Also note the counter-strand: the mindset page’s “value for those that are touched by it”, and the duty not to exploit unwitting subjects.
- §8 tension 11. Qualify it with the CIO chatbot case.
- §8 Gaps. Re-tag “no reported evaluation” as [he says so] (JLME p.567). Reframe “no quantitative treatment” per 18.2. Soften “whistleblowers and employees” to “treated in programme materials, not in his essays”.
- T3 “Development.” Add 2019 applied AI cases: AI Optimize, Trust Me, Predictim, Maven, and the Loss of Agency card.
- Source files. Amend the README and the
_text.mdheader of the OpenAI hypothetical: the entries are ChatGPT’s (see item 12).
19. Digest#
This share is mostly institutional or co-authored material (Nexus pages, cards and case studies, 2019–20; the 2022 counter-influence guide; the 2024 ATP-Bio paper), anchored by a few texts of his own (2020 posts, the NSF submission, Slate, his quoted words in TechTrends). Weighted for provenance, these materials add five things to the map.
First, they settle Andrew’s first note decisively. Between 2019 and 2024, in every provenance tier, risk innovation is described as complementary to conventional risk assessment, never as a replacement. - The Nexus site offers approaches “that complement existing ones”. - In March 2020 he told readers managing COVID-19 that the Nexus website “should not be your first port of call” and to go to the CDC and WHO first, since risk innovation “dovetails into more formal ways of assessing and managing risks”. - The 2022 guide keeps “the likelihood of adverse outcomes” as risk’s simplest form. - The 2024 JLME paper says the approach was “conceived from the outset as complementary” to ERM, SWOT and ISO 31000. - In 2023 he described himself as combining a “physicist mindset, understanding of risk, innovation around how we think differently about risk”.
The map’s C3, T1 and §2 should present quantitative risk science as his foundation, with risk innovation layered where it “runs out of steam”.
Second, they show that the humility in Andrew’s second note is an old, deliberate stance, not a missing capability. - In 2019 the Nexus argued that probability-based framing fails where risks are “not easily quantifiable, or are not associated with clear causative links”, and that such risks are then “pushed to one side”. That is the mechanism of false comfort in methods. - In 2023 he told NSF that dangerous failure modes are “obscured through established – and often outmoded – approaches”, and told TechTrends that “we don’t have theories of advanced technology transitions”. - His tools choose subjectivity to avoid “paralysis by analysis”, handle black swans through “safety net” strategies rather than prediction, and, in the 2022 guide, commit users to own and learn from harm because misuse cannot be fully foreseen.
He does not reject numbers; he rejects letting measurability decide what counts. The map’s “no quantitative treatment of AI risks” gap should be reframed accordingly, though the word “hubris” itself does not appear in this share.
Third, several ideas are older than the map says. - Risk innovation began in March 2013 with Michigan entrepreneurship students. - Orphan risks as known but unfunded, “not necessarily because the risks are not recognized”, and as lacking “agreed upon tools, standards, or mitigations”, is 2019 programme material. That secures for him part of what the map marks [mixed] in 2026. - AI was already an object of his applied tools in 2019: opaque machine-learning decisions as “loss of agency” and lost accountability; the AI Optimize scenario, where delegating decisions to an untraceable system multiplies profit; Predictim; Project Maven. - Compressed disruption timescales (“from years to months”) are in his July 2023 NSF submission. - A benign version of AI reshaping its user (“fine-tuning my brain”) is in his 2023 Slate essay. - In 2021–22 he helped a team apply risk innovation to a chatbot built to form “a trusted relationship” and shift beliefs for a public good, while warning about imposing values. That is a practical root of his 2024 question, “who decides what is good for society?”
Fourth, they sharpen the “whose value?” tension, and show how he speaks to industry. The toolkit’s operative logic is prudential reciprocity, stated outright in the 2022 guide: “threatening stakeholder value becomes a threat to principal agent value”. The JLME paper says its “primary purpose is to help enterprises”. The case studies frame Flo’s harm as perception damage. His pitch to innovators and to government is that responsible practice is how they succeed: “because of … rather than in spite of it”; “win-win”; responsible development as part of national competitiveness in the NSF submission. For the coming article, this shows he engages industry in its own terms, and where that approach thins out: where affected people cannot make their loss felt, reciprocity protects no one, and only duty-based commitments (like the guide’s duty not to exploit the unwitting) remain.
Fifth, provenance corrections.
- The filled-in OpenAI planner (2023) is ChatGPT’s role-play output, not his analysis. Only the exercise design, the mapping of risks onto the planner and his framing post are his. The README and the planner’s _text.md header should be amended.
- Chapter 4 of the 2022 guide is the Lincoln Laboratory engineers’ work.
- The case studies are unsigned programme products.
- He himself names the method’s evaluation gap: the ATP-Bio study “was not designed to provide proof of positive impact”. The map’s gap should be re-tagged as one he acknowledges.