T7. Responsibility, ethics and the people behind technology#

Andrew Maynard on hubris, “could vs should”, myopically benevolent science, technologies of hubris, the honest broker, founders and tech leaders, inequality and justice, and the status quo (2014 to September 2026)

Scope and evidence#

Sources. - The batch digests and notes (notes/B01–B32). - The notes on Films from the Future (book/notes-FFTF-A–F). - concept-index.md and timeline.md. - About 60 posts read directly, with quotes checked against corpus/ and the text of Films from the Future.

What counts as evidence. Only Andrew’s own prose.

Excluded: - AI-generated text. This includes the Perplexity “Top takeaways” in 2025-03-02 and 2025-04-13, and the Deep Research analysis in 2025-03-09 (only his framing and footnotes are used). - Guest posts. - Modem Futura podcast posts, at the user’s instruction.

Weaker evidence: - 2019-08-13, co-written with Garbee. - 2026-07-16, a Fable-drafted paper he rewrote himself; the “argument architecture” is his. - 2026-09-24, his lecture, drafted by Claude and line-edited by him.

Conventions. “FFTF p.X” refers to the printed page of Films from the Future (2018). Where I interpret rather than report, I say so.

1. The position in brief#

Across twelve years, Maynard’s account of who creates technological risk is remarkably consistent. Most of the damage done by powerful technologies traces back to people who mean well. Scientists, engineers and entrepreneurs are rarely villains: “most of them care deeply” (FFTF p.17). They fail for three reasons: - They are absorbed in what they can do. - They trust their own judgment of what is responsible. - They decide for other people what is good without asking them.

Responsibility therefore cannot be self-certified. It needs: - humility; - other people’s expertise, including that of the publics affected; - “checks and balances around who gets to do what” where consequences are widespread or irreversible (FFTF p.166).

Four further commitments give this diagnosis its particular shape.

1. Self-implication. He repeatedly locates the failing in himself: “even myself at times” (FFTF p.220).
2. Admiration for audacity. He finds the visionary leaps of entrepreneurs “exhilarating” (FFTF p.166). He therefore refuses to treat hubris as simply bad.
3. Justice. He asks who benefits, who bears the harm and who decides. The Luddites, in his reading, fought technology’s “socially discriminatory and unjust use” (FFTF p.192).
4. An obligation to innovate. Renouncing technology “from a position of privilege” is itself an injustice (FFTF p.288).

How firmly he holds it. The core diagnosis is among the most stable things he believes: good intentions are insufficient, self-judged responsibility is blinkered, and decisions must not be left to experts. He stated it in 2014–16 and systematised it in the 2018 book. He republished it unchanged in 2023 and in 2025, when he called it “more relevant now than it was then” (2025-03-02 the-lure-of-permissionless-innovation).

What has moved. - His verdicts on particular leaders. - Where he locates the cause. In 2018 it is individual psychology: myopia and hubris. From 2023 to 2025 he names and criticises AI leaders. By 2026 the account is structural: “sincerity almost always operates inside an incentive field” (2026-07-16). - His confidence in the remedies, engagement and responsible innovation. By 2025 he doubts that they can keep pace.

2. Positions and reasoning#

2.1 Sincere people, structural blindness: “myopically benevolent science”#

What he argues. The foundational text is Films from the Future. - In the Jurassic Park chapter he writes that he has met “remarkably few scientists and engineers who would consider themselves to be unethical or irresponsible” (FFTF p.36). Instead he knows plenty who are “so engaged with their work and the amazing things they believe it’ll lead to that they sometimes struggle to appreciate the broader context” (p.36). - InGen’s scientists “are trying to be responsible—at least their version of ‘responsible’—but are tripped up by what they don’t know, and what they don’t care to find out” (p.38). - The Man in the White Suit gives him the figure he returns to most. Sidney Stratton “never thought to ask anyone else what they wanted or needed” (p.211). - He names the pattern myopically benevolent science (section title, p.218), “benevolent myopia” (p.220) and “general benevolence and specific self-interest” (p.221): “too many well-meaning scientists presume to know what society needs, without thinking to ask first” (p.222).

The self-implication is deliberate and structural to the argument. - He confesses that in his own aerosol research, “I was quite happy to coopt a narrative of social good so that I could continue to satisfy my scientific curiosity” (FFTF p.219). - He sees Stratton’s traits “in many of my peers, and even myself at times” (p.220). - He concludes that “All of us, it has to be said, have a bit of Sidney Stratton in us” (p.227).

The evidence is his own teaching. He ran an entrepreneurial-ethics course at Michigan from 2013. His students “had good intentions but no idea how to translate them into good practice”, and their values were “naive and ephemeral” (2019-04-15 tech-companies-need-an-ethics-reset). The book’s classroom profit game shows “how hard it is to translate good intentions into good actions” (FFTF p.221).

The remedy. Responsible science “is about more than just having good intentions”. It also requires “the humility to recognize your limitations, and the willingness to listen to and work with others” (FFTF p.39).

Why he holds it. The reasoning is epistemic, not moralising: - complex systems defeat any single perspective; - blind spots are built into expertise itself; - self-interest quietly bends general benevolence.

He applies the diagnosis to the AI sector without softening it. AI development is “driven by a small group of experts and companies who believe that they have all the understanding they need” (2023-10-02 responsible-ai-lessons-from-nanotechnology).

The latest form of the argument (2026) generalises it from minds to institutions. The frontier labs’ pattern of dropping hard-to-measure risks is “neither accidental nor, for the most part, cynical”. Their framework authors have “spent their careers trying to make powerful technologies safer”, but “sincerity almost always operates inside an incentive field” (2026-07-16; mixed provenance, see Scope). He draws the lesson from his Garbee work: “Innovation cultures that sincerely want to do good nonetheless tend to reject frameworks that arrive as top-down obligation” (2026-07-16).

The same refusal to assume bad faith extends to emergent AI harm. Character.AI chatbots may harm users “not because the company is necessarily acting irresponsibly” but because influence is an emergent property (2024-10-27 personal-ai-chatbots-and-stochastic-agency).

Firmness. Very firm and very central. This is his basic model of developer psychology, and it is the premise on which much else in the thread rests.

2.2 Hubris and “technologies of hubris”#

The concept and its double edge. In the Ex Machina chapter, under the section title “Technologies of Hubris”, he defines hubris as “that excessive amount of self-confidence and pride in one’s abilities that allows someone to see beyond seemingly petty obstacles or ignore them altogether” (FFTF p.166).

Crucially, hubris is also how things get done: “This is how transformative technology happens: not in slow, cautious steps, but in visionary leaps” (p.166). Of Musk’s Mars plans he writes, “I must admit that I find this exhilarating” (p.166). The problem is that such leaps “often come with a massive risk of unintended consequences” (p.166). Nathan Bateman is the type: “tech-savvy, but socially ignorant” (p.163), with “a very one-dimensional brilliance” (p.166).

Why it is more dangerous now. He gives a short history of recoverability: - Pre-industrial mistakes could be left behind. - From the Industrial Revolution on, changes became “hard-to-reverse”. - In “the nuclear and digital age”, consequences outrun containment.

So hubris and permissionless innovation are “playing with fire in a world made of kindling” (FFTF p.167).

Other forms of hubris in the book. - Designed worlds. Cyberspace “reflects all the limitations and biases and illusions that come with human hubris” (p.146). - Collective hubris. The Day After Tomorrow is “a warning against human hubris” at the scale of a civilisation (p.260). - Humility, and its limits. Humility is the antidote. The 2015 AI warnings from Musk, Hawking, Gates and others were “a rare display of humility in a technological world where hubris continues to rule” (p.167). “But humility alone isn’t enough”: it must be joined to plausibility (p.168). - Vocabulary. In the early chapters the word “hubris” is often absent. The idea is carried by “the folly of entrepreneurial arrogance” (p.26) and by “delusions of power” (p.43).

Later uses. - Engagement and progress. Scientists “hubristically ignoring the very communities” they claim to help (2023-09-04 why-public-engagement-is-so-important). GMOs as “a masterclass in how naivety, hubris, greed, and a lack of broad engagement, can create near-insurmountable roadblocks to progress” (2023-10-02). Reading: here hubris damages the innovators’ own cause as well as the public. - AI labs. He contrasts Anil Seth’s humility with “hyperbolic and hubristic proclamations coming from proponents of advanced AI”. He objects to the idea that “it’s OK to simply go fast and break things because we’re building something new” (2024-06-30 seth-is-conscious-ai-possible). - Colossal Biosciences. “The hubris here is palpable”. AI companies are “full of themselves”, “but Colossal takes this to a whole new level. You have to admire their audacity though!” (2025-04-13 de-extinction-conservation-futures). He also allows that Colossal “may be onto something — at least technologically”. - His own field. He asks how often we overlook what matters “simply because we are too arrogant to look beyond the narrow confines of our own hubris?” (2025-03-09, his footnote).

By 2026 the concept has become symmetrical. - Academia. He argues for “helping ensure student success comes before academic hubris” (2026-03-29 can-ai-create-an-undergraduate-degree-plan). - Refusal can be hubris too. Using the Sorcerer’s Apprentice, he writes that Mickey “acted with certainty unshaken by his ignorance”. He then asks whether “trusting that traditional mastery alone is enough” is the modern equivalent (2026-04-11 ten-questions-about-ai-and-higher). - Himself. He called his own 2023 title “a little hubristic” (2023-11-26 everything-youve-heard-about-ai-risk-is-wrong). In 2026 he entertains, then rejects, the thought that he is “an anachronism … lost in his own myopic hubris” (2026-07-19 publish-or-perish-ai-vs-human-vs-human).

Reading: for Maynard, hubris is at root an epistemic failure, certainty outrunning understanding. It is not simply vanity. That is why he can apply it to entrepreneurs, to civilisations, to AI refusers and to himself.

2.3 “Could vs should” and the ethics of permission#

Could vs should. Ian Malcolm’s line (“so preoccupied with whether or not they could, they didn’t stop to think if they should”) is his most-used touchstone. It is quoted, not his own, but framed repeatedly. His own gloss is that “the more complex the science and technology we begin to play with is, the more pressing this distinction between ‘could’ and ‘should’ becomes” (FFTF p.39). Early computing built cyberspace on “what we could do rather than what we should” (p.144–145).

He applies it widely: - Neuralink: companies “so wrapped up in what they can do, that they lose sight of the ethics behind what they should do” (2019-07-23 neuralinks-technology-is-impressive). - The Tesla Bot: “Just because he can … who’s to say that he should?” (2021-09-07 should-we-be-worried-about-elon-musks-tesla-bot). - Gene editing: 2022-06-13 jurassic-park-dominion. - His teaching: in class he frames the question as the line between what entrepreneurs “can” and “should” do (2023-08-28 dear-mr-musk-can-i-have-my-book-back).

Reading: the line opens questions rather than settling them. The analytic work is done by his treatment of permission.

Permissionless innovation. He defines the idea more broadly than Adam Thierer’s policy doctrine: “innovation that is conducted in the absence of permission from anyone it might impact” (FFTF p.159).

His distinctive refinement is that “permissionless innovation isn’t necessarily reckless innovation”. Rather, it is innovation “that the person doing it thinks is responsible” (p.162). The flaw is self-certification: “With the best will in the world, a single innovator cannot see the broader context” (p.162). Innovators are trapped in their own Plato’s Cave. Nathan’s downfall is that “he had no translator between himself and a bigger reality” (p.163). Hence “sometimes, constraints and permissions are necessary” (p.163).

The lure, and its value. He grounds the lure of permissionless innovation in his own experience. Of a PhD all-nighter spent bending lab rules he writes: “it’s shocking how quickly I sloughed off any sense of responsibility to get the data I needed” (p.161). He also names the “glitch” in his own argument: without a “gung-ho attitude”, innovation and its good would be “much, much slower” (p.163). “Too much blind speed, and you risk losing your way. But too much caution, and you risk achieving nothing” (p.163).

How the concept hardens over time. - 2019. Scare-quoted as “‘permissionless’ innovation that’s driven by power, wealth and a lack of accountability” (2019-01-16 responsible-innovation-entrepreneurs; also 2019-03-07). - 2023. He diagnoses the US as having “an ethos of go fast, break things, and don’t worry about the consequences” (2023-10-30 white-house-goes-all-in-on-responsible-ai). - 2025. He sees it winning politically: the AI game is increasingly “to go fast and break things in the hope that someone else will clean up the mess” (2025-02-23 evo-2-dna-ai). He adds an operational test in a new footnote to the reposted chapter. Experimenting in low-risk, reversible, linear systems is fine. Breaking “people, governance, society, and the planet” is not (2025-03-02). - The 2025 Action Plan. He reads its “try-first” culture as ask-forgiveness-not-permission, “go fast and bugger the consequences” (his words, flagged as such). Responsible innovation is “actively portrayed as a barrier” (2025-07-23 americas-ai-action-plan).

A related failure is diffused responsibility. Transformative technologies “creep under our collective societal skin” through eager adoption and “an assumption that any potential issues will be handled by someone else”. Universities risk treating responsible innovation as “somebody else’s problem” (2023-04-18 universities-need-to-be-investing).

Firmness. Firm and growing firmer. His concession on speed is constant, but after 2023 it carries less weight.

2.4 Moral certitude, the right to act unilaterally, and the honest broker#

Immoral logic. The Inferno chapter takes the benevolent-myopia diagnosis to its extreme: the “darker side of technological innovation … when good intentions lead to seemingly logical, but not necessarily moral, actions” (2018-10-18 the-honest-broker-meets-dan-browns-inferno; FFTF p.231). - He flags “a continuing theme in this book”: “powerful entrepreneurs who not only have the courage of their convictions, but the means to act on what they believe” (p.232). - He worries about “a world where powerful individuals are able to translate their moral certitude into decisive action” (p.237). - Reasoning from extrapolated catastrophe has the “same conceits we see in calls for action based on technological prediction”. It creates “artificial certainty”. It puts ideology ahead of social responsibility. It assumes the “enlightened” may act without consent (p.240).

The concession that sharpens the point. He admits that book-Zobrist’s outcome “intrigues me”, and that a lone visionary “could probably do it better than a committee” (p.248). The question stands regardless: “where do they get the right to act unilaterally on issues that ultimately impact us all?” (p.249). Reading: this is his clearest statement that legitimacy rests on consent and inclusion, not on good outcomes.

The honest broker. From Roger Pielke Jr.’s four roles he chooses the Honest Broker: “This is the role I try to carve out for myself in my public-facing work” (FFTF p.246). The role “avoids mistaking personal values for the ‘right’ values” (p.246).

He qualifies it at once. Where silence becomes tacit support for inaction (his examples are climate, nuclear weapons and vaccines), advocacy is warranted. Even then “it’s hard to justify one person being the sole arbiter of truth” (p.247), so it should run through institutions and “collective advocacy” (p.244).

He recommends Pielke again in 2023, on “the fine line between research, social responsibility, and activism” (2023-09-15 weaponizing-the-genome). He describes his own writing in the same spirit: “I do not write pieces that are polarizing, that are preachy, that push an ideology or an agenda” (his own prompt text in 2025-01-30 ai-at-a-crossroads).

The “fix” frame as benevolent certitude. From 2023 the Inferno logic returns in a mild, well-meaning register. - Musk’s truth-seeking AGI: “an assumption of ultimate social ‘truth’ is more often an excuse to impose an ideology or worldview on others” (2023-07-19 elon-musk-maximally-curious-agi). - Neuralink’s Blindsight shows “savior complex-like aspirations” and “the troublesome idea of ‘fixing’ people” (2024-09-18 neuralink-blindsight-brain-computer-interface). - If AI can fix anything, that must include “‘fixing’ people” (2024-10-06 the-double-or-nothing-bet-on-ai-fixing-the-climate, footnote). - Even Amodei’s humane essay leaves it “unclear who decides what is ‘normal’ and what needs to be ‘fixed’” (2024-10-13 amodei-machines-of-loving-grace).

Reading: this is the same objection as in Inferno. The danger is certainty about others’ good, backed by the means to impose it.

Firmness. Firm on consent and against unilateral action. The honest-broker self-identification is explicit but book-rooted. He has not revisited it by name since 2023 (see §6).

2.5 How he characterises founders and tech leaders#

The early template (2016–2019): technically brilliant, socially naive. - Musk’s Tesla plan: it’s “not clear yet whether they have the social and political savvy to make it work”. He adds that “numeric logic is often trumped by what we intuitively think and feel is important” (2016-03-12 itll-take-more-than-tech-for-elon-musk). - SpaceX: Musk must be “as socially adept as they are technically talented”. Success depends on “whether society writ large grants SpaceX and Elon Musk the freedom” to proceed (2017-04-10 dear-elon-musk-your-dazzling-mars-plan). - Schwab’s book “reads as if it were written in a vacuum” (2016-01-11 the-fourth-industrial-revolution).

His remedy in these years is partnership with his own field, and he discloses the interest: “I’m admittedly a little biased” (2016-03-12).

The book’s type-figures. - The mega-entrepreneur (Hammond). - Nathan, “a wealthy and unaccountable entrepreneur who is supremely confident in his own abilities” (2019-01-16). - “a new type of entrepreneur … who has substantial power and drive to change the face of technology innovation” (FFTF p.161). - NewSpace leaders who “aren’t answerable to social norms and expectations” (p.165).

Silicon Valley entrepreneurialism. - Of entrepreneurs: “Not only do they not care for established ways of doing things”, they exploit the cracks between disciplines (2020-10-15 the-ethics-of-advanced-brain-machine-interfaces). In the same post he admits “we were somewhat naïve” about Neuralink’s medical priorities. - In 2021 the worry becomes political: “we risk handing our futures to innovators whose vision exceeds their understanding” (2021-09-07). - Neuralink in 2024 is “caught up in a cult of personality”, promising superhumans “locked into a subscription plan” (2024-01-30 first-in-human-trial-of-neuralink-bci). - Yet he is even-handed: “I’m not begrudging this approach”, and “the reality distortion field is real, and I suspect that critics will discount it at their peril”. He adds the caveat that Neuralink comes from a sector where “monetizing the customer is the name of the game” (2024-03-21 elon-musks-neuralink-plays-mind-games).

AI leaders: individual verdicts that shift. - Altman. - In 2023 he “comes across as genuinely sincere” in asking to be regulated. The “cynical” reading, an “exclusionary playing field”, is not happening, “at least, not yet” (2023-05-17 ai-senate-hearing-may-2023). - By November 2023 Altman is policymakers’ “shiny person” (2023-11-18). - Over the Johansson voice in 2024, Altman “arrogantly plowed ahead”. The episode shows “childish irresponsibility” and “disdain for the dignity and rights of individuals” (2024-05-21 openais-problem-with-the-movie-her). - Schmidt. The “leave it to the technical experts” mentality “never plays out well”. But “In fairness … I suspect that he has a more nuanced perspective” (2023-05-15 erik-schmidt-ai-regulation). - Sutskever. He complains of “people who assume a mantle of expertise on safety but have no idea of what they’re talking about”, yet grants that SSI is “trying hard to do good here” (2024-06-20). - Bostrom and FHI. Bostrom favoured “philosophical elegance” over physics. His ideas “fueled Silicon Valley’s ‘tech bro’ culture” and reflect “a disdain for society”. Future-building should not be “left to an elite group of thinkers and innovators” (2024-04-28 beyond-the-future-of-humanity-institute). - Andreessen. His manifesto indulges technological “foreshortening” and begs the question “who decides who will suffer and who will thrive” (2023-10-19). - Amodei. Praised as “tempered with humility and reason”, but corrected on the “fix” frame and the “deficit model”, “debunked decades ago” (2024-10-13). - Musk. - He writes that “for all his many flaws, I can’t help but be inspired sometimes by the audacity” (2023-07-19). - Musk matters for “not how wild the ideas are, but how much influence they have”. Maynard confesses “I get where he’s coming from”, while hoping for “an informed counterbalance” if Musk shapes policy (2024-08-04 7-key-takeaways-from-elon-musk-and-lex-fridman). - By 2025 he calls his own 2018 view “a rather naive perspective on Elon Musk”, and DOGE “a rather naive and uninformed application of permissionless innovation” (2025-03-02).

The sector as a whole. - The belief that tech leaders’ good intentions will yield beneficial AI is “sheer fantasy” (2023-12-15 pope-francis-artificial-intelligence). - In his words, “a bunch of tech bros forget everything they ever knew about the Dunning-Kruger effect when it comes to governance and policy” (2024-12-29 fantasy-top-ten-lists-2025). - Companies are “trying hard” but “lack the breadth of vision and understanding” (2025-01-07 universities-need-to-step-up-their-agi-game). - Labs are “mission-driven, often allergic to imposed process, and rarely short of conviction in their own exceptionalism” (2026-07-16). - Developers act “as if they’re the first people to notice” long-known risks (2026-09-15 will-ai-really-kill-us-all). - Companies lack the breadth “to be able to decide for humanity” (2026-09-24).

Founders outside AI. - Colossal (§2.2). He had already noted in 2022 that de-extinction’s “technology and the entrepreneurial drive” echo Jurassic Park (2022-06-13). - Orchid, on embryo screening: “I worry about the social and ethical risks. A lot. But I also respect what Siddiqui and Orchid are trying to do”. He asks for “neither demonizing them or raising them, god-like, on a pedestal”, but “hard but respectful dialogue” (2024-04-14 welcome-to-the-age-of-swipe-and-select-embryos).

Reading of the pattern. He grades leaders on three things: - Humility. He praises Joy, Seth and Amodei, and faults Nathan, Colossal and Bostrom. - Social understanding. Entrepreneurs are “so enamored with cool tech that they fail to spot the social messages” of the films they love (2024-05-26 should-tech-entrepreneurs-be-banned-from-scifi). - Respect for consent and dignity. His harshest language, used about Altman in 2024, is reserved for overriding a person’s explicit refusal.

He engages leaders because of their influence, and he rarely alleges bad faith. His warmth and his criticism coexist, sometimes within weeks of each other (see §6).

2.6 Experts, elitism and who decides#

Scientists’ own responsibility. His earliest posts turn the critique on his own profession. - Exaggeration. Research press releases compromise the “chain of trust between the researcher and reader”. He wants a culture that “placed the health and well-being of their constituencies before their own aggrandizement” (2014-12-14 researchers-should-take-more-responsibility-for-exaggeration). - Elitism about citizen science. The lingering elite attitude that nonscientists “should revere, but not interfere with, science” is his target. When he suggested scientists listen to ordinary people, senior advisers told him this would be “a really bad idea” (2016-01-12 can-citizen-science-empower-disenfranchised-communities). In the book version, a PCAST member replies, “That sounds like a very bad idea” (FFTF p.222). - Universities. Individual academics have consciences, but “the institution gets in the way” (2016-01-31 public-universities-must-do-more). - Nanotechnology. “Brand-nano” “fudged the science to sell the idea”, and experts “went along with it because of the promise of funding”. This is a field he helped build (2018-02-21 the-bs-and-the-science-of-nanotechnology).

Publics as holders of expertise. - “Scientists and technologists don’t have a monopoly on expertise and insights”. Most people have “a pretty high level of expertise in what’s important to them and their communities” (FFTF p.222). - Refusing to engage leads to “scientific elitism, and ignorance that’s shrouded in arrogance” (p.223). - An AI executive’s “let’s not talk” approach is high-risk, epistemically poor and ethically suspect. There is “a moral imperative to engage broadly when a technology has the potential to impact society significantly” (p.227). A developer “probably shouldn’t have complete autonomy over deciding what you do” (p.227). - Leaving these questions to scientists, innovators and politicians is “an abdication of responsibility” (p.288). “We all become stakeholders” (p.282). - Public apathy itself hands the field to “those that do care”, who may “do what they like, even if it ends up harming us” (p.286).

The same argument, applied to AI. - “It’s complicated” is not an excuse for excluding people. People “don’t need to understand the inner workings of AI” to judge what it threatens (2023-05-15). - The “too complicated” claim shows “a very real risks [sic] of hubristically ignoring the very communities” technologists claim to help. Developers “tout the importance of public engagement in the abstract, but strenuously resist it in practice” (2023-09-04). - “Loud (but not necessarily informed) voices” fill the “insights vacuum” (2023-04-10 as-ai-goes-to-washington). - AI for good “cannot simply be left in the hands of tech entrepreneurs and AI experts” (2023-12-22 un-governing-ai-for-humanity). - Responsibility “isn’t just a problem for companies to fix, or for policy makers to govern” (2025-08-31 holding-on-to-our-humanity-age-of-ai).

Firmness. Constant and emphatic. What changes, in 2025–26, is his view of who should act. Companies lack breadth. Governments are slow. Civil society lacks “the wherewithal to lead”. Publics are “critically important” but cannot be handed the problem. Universities should step in as “an accelerator and a catalyst” (2026-09-24).

2.7 Inequality and justice#

Justice is the moral axis of the thread, not an add-on. Five strands recur.

1. Technology amplifies power. - Technologies “in the hands of a few people can amplify the power they have over others” (FFTF p.110). - “The future is designed by the powerful”, and without inclusive design principles the default is “a future of the powerful, by the powerful, for the powerful” (2019-03-31 design-principles-for-de-marginalizing-the-future). - Wealthy visionaries “impose their visions of the future on us” (2021-09-07). - “even with good intentions, bad things happen when you mix powerful technologies with power-hungry people” (2022-06-13). - “Unelected billionaires” (2024-01-17 ai-global-risks-2024-wef-davos). - The AI Action Plan “prioritizes power before people” (2025-07-23).
2. Harm lands on the least powerful first. His occupational-health past grounds this. - Scientific doubt over black lung was “an uncertainty that suited the mine owners” (FFTF p.120). - New technologies harm “the first tier of people” who make them (p.121). - Preventing a disposable workforce is “the responsibility of everyone, not just employers or regulators” (p.122). - In 2026 he adds that the channels through which harm reaches firms (backlash, litigation, regulation) “are not equally open to everyone” (2026-07-16). - Within institutions, power differentials matter too: the advisor “who holds your academic career in their hands” (2025-10-26 ai-misuse-in-student-advisor-collaborations).
3. Markets do not deliver equity. - “It’s too easy to assume that technology trickle-down is a given” (FFTF p.101). - “Free-market economies can thrive on social inequity and injustice” (p.103), so “a system of checks and balances” must steer markets (p.103). - “Being smart doesn’t make you good” (p.108). - Free intelligence is not free: “Somebody is paying somewhere” (2026-09-24, Q&A footnote).
4. Luddites and critics reclaimed. - The Luddites were “not fighting against technology, but against its socially discriminatory and unjust use” (FFTF p.192). The goal is technology “for the benefit of all, not just a privileged few” (p.193). - “Neo-Luddite” is “usually a term of derision and censorship” (p.191). The “Luddite” slur expresses “a worldview that all technological advances are ultimately good for society” (2023-05-12 unraveling-the-luddite-narrative). - He defended Musk, Gates and Hawking against ITIF’s award: “If that’s what it means to be a Luddite, count me in!” (2018-12-15 if-elon-musk-is-a-luddite-count-me-in). - In 2026 he notes that talking about AI risk “gets you branded as a technology-pessimist, or even a Luddite” (2026-05-10 do-not-do-this-with-ai).
5. The individual against the aggregate. - The deepest harm is a technology that “makes us forget the worth of others” (FFTF p.62). - People justify “the unethical and the immoral in the service of an assumed greater good” (p.84). - Technological foreshortening says in effect “some people will inevitably suffer as we strive to build someone else’s dream of a better future, and that’s OK” (2023-10-19). - The hard concept of care rejects those “who have little time for the individuals who make up that aggregate” and endorses Tronto’s “privileged irresponsibility” (2025-03-09). - LLM “LinkedInification” most exposes “anyone who lives on the edge of convention” and “degrades people” (2026-03-08 ai-linkedinification).

The counterweight is also framed as justice. Renouncing technology “from a position of privilege” risks “denying too many people without the same privileges the chance to make their own decisions” (FFTF p.288; republished 2019-03-06). Past inventions look good “especially if you’re privileged and well-off” (2018-12-15). Reading: equity pulls him both ways. It argues against unequal harm, and against abstention by the comfortable. This is why his stance is steering rather than stopping.

2.8 The status quo#

His treatment of the status quo is two-sided, and consistently so.

Respect it: innovators underestimate it at their peril. - The White Suit planetary-protection anecdote: the scientist “deeply underestimated the inertia of the status quo” (FFTF p.224). - “The harsh reality is that discovery never happens in isolation” (p.225). - Resisters are not ignorant. Everyone is “shrewd enough to see how change supports or threatens what they value”, and they fight to protect it (p.225). - Nano pants showed how “seemingly beneficial inventions can sometimes threaten the status quo” (2018-10-11 a-nanoscale-perspective-on-the-man-in-the-white-suit). - The SpaceX social-licence argument (2017-04-10) and the risk-as-threat-to-value framework are this insight made operational. - He condemns anti-technology violence and the “lazy” narratives that legitimise it (2024-02-18 setting-fire-to-self-driving-cars-is-bad). Respect for resistance is not a licence for it.

Don’t worship it: defending it is also a risk. - In 2016 he criticises how, when novel technologies arise, “the overwhelming impulse is to maintain the status quo” by shoehorning them into old regulatory frameworks (2016-01-11 thinking-innovatively-about-the-risks-of-tech-innovation). - Resilience “is not necessarily about maintaining the status quo, but about protecting and preserving what is considered to be ‘of value’” (2018-10-25; FFTF p.262). - Entrepreneurs who have “the audacity to buck the status quo” change history (FFTF pp.165–166, reposted 2025-03-02). - Defending the status quo robs people of the chance to adapt “in ways that could benefit everyone” (FFTF p.225). - In 2025–26 the target is his own institution-type. Universities are “guardians of the past more than leaders toward the future” (2026-08-30 do-universities-have-a-place-in-bill). The academic “crab bucket” pulls down those who climb out (2026-09-24). Education risks “sacrificing what could be on the altar of a blind devotion to what is” (2025-03-30 reimagining-education-in-an-age-of-ai).

Reading: the criterion that reconciles these is value, not stability. Neither innovators nor incumbents may treat their own position as self-evidently right.

3. How the thread developed#

Constants. - Good intentions are not enough. - Responsibility cannot be self-certified. - Publics have relevant expertise. - Consent and legitimacy come before outcomes. - Justice is the test. - Developers are not demonised. - He admires audacity and implicates himself.

Shifts. 1. The cause moves from individual psychology (2018) to structure (2026). 2. Permissionless innovation goes from a risk to be managed to the prevailing politics (2025). 3. His confidence in engagement and responsible innovation turns to doubt (2024–25). 4. Critique of innovators becomes critique of hubris on both sides, including refusers (2026). 5. Personal judgments of named AI leaders sharpen (2024–25), then partly give way to structural explanation (2026).

4. Key concepts (his terms)#

5. Connections to his other threads#

• Risk as a threat to value; risk innovation; orphan risks. The justice and status-quo arguments are his risk theory seen from the human side. Resisters protect “what they value” (FFTF p.225). Social licence depends on whether people feel something they value is threatened (2017-04-10). Orphan risks and the Risk Innovation Planner turn “ask those affected” into an enterprise tool, so responsibility becomes enlightened self-interest. The 2026 paper joins the two explicitly, and flags the limit that conversion channels are unequal.
• Responsible innovation and governance. Myopic benevolence is why inclusion and reflexivity matter. His preference for culturally embedded norms over top-down rules follows from his view that sincere entrepreneurs reject imposed obligation. His declining confidence in responsible innovation (2024–25) weakens the main remedy this thread relies on.
• Complexity and irreversibility. Hubris is dangerous because of the “world made of kindling” (FFTF p.167). The 2025 reversibility test for permission is a complexity argument.
• Plausibility. Immoral logic rests on “artificial certainty” (FFTF p.240). The same critique runs through his scepticism about superintelligence ideology (2024-04-28) and about violent anti-technology action (the ITS bombings).
• Manipulation and benevolent persuasion. Several of his cognition concerns are the Inferno and “fix” critique applied to minds:
• “But who decides what is good for society?” (2024-09-01 is-chatgpts-new-voice-mode-dangerously-persuasive);
• paternalism in the “economic gradient” (2024-07-13 ai-choice-engines-sunstein);
• machines “that know us better than we know ourselves” (2019-01-16).
• Being human, worth and dignity. The “convenient lie” that some beings are less than human (FFTF p.57) extends to AI moral status. He resists “the economic expediency of denying consciousness” (2023-08-23 could-we-build-conscious-ais-in-the-future). The same dignity lens shapes his response to the Johansson episode.
• Universities. Both the earliest (2016) and the latest (2026) versions of his institutional critique are about universities: public good versus prestige, the crab bucket, “what we owe our students”, and duty of care (2025-11-09).
• AI in his own practice. He extends self-implication to his own AI use. He was “suckered by Claude” (2026-02-08). He deliberately wrote the Action Plan analysis without AI (2025-07-23). He wonders whether his own judgment is “myopic hubris” (2026-07-19).

6. Tensions, ambiguities and gaps#

1. Admiring audacity while criticising hubris. - He finds entrepreneurial leaps “exhilarating” and concedes the “glitch” that caution slows good innovation (FFTF pp.163, 166). - He admires Colossal’s “audacity” in the same breath as condemning its hubris (2025-04-13). - The 2025 reversibility test is his only explicit criterion for when a leap is acceptable. It puts AI, BCIs and gene editing on the wrong side, without saying what should follow.

2. Charity towards developers, and moral censure of them. - His default is charitable: he grants sincerity, adds “In fairness”, and asks for “neither demonizing”. - Yet Altman in 2024 is arrogant, childish and disdainful, and FHI shows “a disdain for society”. - The 2026 structural turn (the “incentive field”) partly reconciles the two, but it thins out individual accountability. He does not say when an actor whose conduct is structurally explained is still culpable.

3. Critic and partner. - He criticises capture and “good intentions” thinking. - He also embraced the ASU–OpenAI partnership (early 2024, per notes), treats Anthropic’s constitution respectfully, co-wrote an AI-assisted book with a venture capitalist, and uses frontier models as research partners. - He discloses interests (“I’m admittedly a little biased”, 2016-03-12), but does not analyse his own entanglement. Reading: his incentive-field model would apply to him too.

4. The honest broker and growing advocacy. - The role he chose is to inform rather than dictate, and to advocate only through institutions (FFTF pp.244–247). - His 2025 Action Plan critique and his 2026 statements read more like issue advocacy. On whether regulation could “simply close the gap” he is “not optimistic” (2026-07-16). Universities must “wake up” (2026-09-24). - The institutions the book relied on to carry advocacy are ones he now finds slow or captured. - He has not revisited the honest-broker framing since 2023.

5. Anti-elitism and a new expert class. - Technical elites have no “monopoly on expertise”. - Yet he urges listening to those “who actually know about responsible innovation” (2025-08-31) and to “people and institutions who know a thing or two about risk” (2026-09-15). - By 2026 publics are “critically important”, but “you cannot hand a problem of this magnitude over to everyday people”, and universities should lead (2026-09-24). - Mechanisms that would give publics real power, as opposed to engagement formats, are thin throughout.

6. Two equity arguments that can pull apart. The privilege critique of renunciation (FFTF p.288) and the justice critique of unequal harm (FFTF pp.100–122; 2023-10-19) both appeal to equity. He never says how to weigh them when they conflict.

7. Enterprise framing and justice. - Much of his practical work casts responsibility as enlightened self-interest: show a firm “a threat to something it values”. - He concedes in 2026 that the harmed can make firms feel harm only through channels “not equally open to everyone”. Beyond disclosure tools, he offers no remedy.

8. Permission and inevitability. - He wants checks on “who gets to do what”. He also increasingly holds that innovation cannot be stopped: “guide and steer” (2025-03-30), and “We can’t pause it” (2026-09-24, which he flags as possibly flawed). - If permission cannot be withheld, it is unclear what the checks consist of beyond channelling, capacity-building and care. - His 2024 conditional pause on companion bots is the exception.

9. Respecting or overcoming the status quo. - He defends resisters as rational protectors of value, yet criticises universities and regulators for clinging to the status quo, and by 2026 calls academic AI refusal a form of hubris. - In principle his value criterion reconciles the two. In practice, whose value counts is the same “who decides” question he presses on others.

10. Gaps. He says relatively little about:

    • present-day labour and collective organising (his cases, the Luddites and the miners, are historical);
    • race and colonial power, beyond predictive policing and some co-written work;
    • the Global South;
    • investors’ duties;
    • whistleblowers and employees inside firms (the “quiet voices” of 2023-04-10 are the main exception);
    • legal liability, apart from deepfakes (2024-02-25);
    • wealth concentration as a structural cause rather than a named risk (“unelected billionaires”).

7. The most important sources for this thread#

1. FFTF Ch.10, The Man in the White Suit (pp.207–229; excerpt 2018-10-11; show notes 2023-09-08). Myopically benevolent science, self-implication, “It’s good to talk”, the inertia of the status quo.
2. FFTF Ch.8, Ex Machina (pp.159–168; reposted 2023-04-16 and 2025-03-02 the-lure-of-permissionless-innovation, with 2025 footnotes). Permissionless innovation as self-certified responsibility, technologies of hubris, the lab confession, the reversibility test, the revision of Musk.
3. FFTF Ch.11, Inferno (pp.230–249; 2018-10-18). Immoral logic, moral certitude with means, the honest broker, “the right to act unilaterally”.
4. FFTF Ch.2, Jurassic Park (pp.27–45; reposted 2025-04-13 de-extinction-conservation-futures, with the Colossal framing). “Their version of ‘responsible’”, could/should, the responsible use of power.
5. FFTF Chs.5–6, Limitless and Elysium (pp.85–127). Privileged technology, markets and inequity, the disposable workforce, first-tier harm.
6. 2019-04-15 tech-companies-need-an-ethics-reset. Good intentions versus practice, from his own teaching; ethics boards as “smoke-and-mirrors”.
7. 2023-05-15 erik-schmidt-ai-regulation (with 2023-09-04 why-public-engagement-is-so-important). No abdication to technical experts; the PCAST anecdote; engagement as a right.
8. 2023-10-19 marc-andreessen-ditch-sustainability. Technological foreshortening; “who decides who will suffer and who will thrive”.
9. 2024-05-21 openais-problem-with-the-movie-her. His sharpest named criticism: consent, dignity, “childish irresponsibility”.
10. 2024-04-28 beyond-the-future-of-humanity-institute. Speculative elites, “tech bro” culture, future-building as collaborative and not elite.
11. 2024-10-13 amodei-machines-of-loving-grace (with 2024-09-18 and 2024-10-06). The “fix” frame, who decides normal, the deficit model, and generous engagement with a humble leader.
12. 2021-09-07 should-we-be-worried-about-elon-musks-tesla-bot. “Whose future?”, “innovators whose vision exceeds their understanding”.
13. 2025-03-09 the-hard-concept-of-care-in-technology-innovation (his framing and footnotes only). Aggregate against individual, privileged irresponsibility, hubris.
14. 2026-07-16 orphan-risks-frontier-ai-maynard (mixed provenance). “Sincerity almost always operates inside an incentive field”, labs’ “conviction in their own exceptionalism”, unequal conversion channels.
15. 2026-04-11 ten-questions-about-ai-and-higher (with 2026-09-24 being-an-academic-in-an-age-of-ai). Two-sided hubris, companies unable “to decide for humanity”, the academic crab bucket.