B20 notes: 2024-11-06 to 2025-01-12 (18 posts)#

Reading notes on Andrew Maynard’s Substack posts in batch B20. Only his own prose is treated as evidence. Quotes are exact, including his original typos, italics markers and curly punctuation.

Batch context: from Trump’s election victory (5 November 2024) to mid-January 2025. OpenAI runs its “12 days of OpenAI”, releasing o1 in full, Sora to the public and Canvas, and then announces o3. Google ships Gemini 2.0, and Apple Intelligence rolls out. Sam Altman’s Reflections predicts AI agents will “join the workforce” in 2025. Maynard is at ASU (Future of Being Human initiative). He is co-hosting the new Modem Futura podcast with Sean Leahy, launched in October 2024, and working in the ethics and policy pillar of the NSF ATP-Bio Engineering Research Center.

Podcast posts. The user asked that the Modem Futura podcasts be left aside. Five posts are Modem Futura promos or episode notes (2024-11-06, 2024-11-12, 2024-12-22, 2024-12-31, 2025-01-07 Cady Coleman). The episodes were not listened to or analysed. Four of the five get one line each. The exception is 2024-11-06 ai-in-a-world-of-trump. It has a few paragraphs of Maynard’s own written framing on AI governance under Trump, which is relevant in its own right, so it gets a short entry. That entry covers only his written text, not the episode.

Relevance summary:

HIGH#

2024-12-17 · navigating-the-challenges-and-opportunities-of-advanced-biopreservation-technologies · “Navigating the challenges and opportunities of technologies that “stop biological time”“#

Provenance. His own prose throughout. The section “Advanced Biopreservation and Risk Innovation — the quick(er) version” is his own summary, taken from his October 2024 talk, of a co-authored paper: Maynard, Oye, Scragg, Tripp and Wolf, JLME 52(3), 2024. The post also quotes the guest editors’ introduction (Wolf, Pruett, Uygun) and a multi-author framing paper he co-wrote (“we further re-enforce this”). Those quoted lines are collective or other people’s text, not his prose.

Argument in his terms. - What is at stake. Biopreservation that can “stop biological time” (organs, tissues, cells, aquatic embryos) could be a “game changer”. It would also raise “a mountain of ethical and social challenges” between capability and “successful and beneficial use”. He calls this a “complex risk landscape”. - Why a technology centre should care about ethics. ATP-Bio’s leaders saw that ignoring the social landscape “would be tantamount to setting themselves up for failure”. He makes two points: - Ethics is part of the reason the technologies exist at all, since they are meant to save lives. - Ethics is also a condition of their success. - Pitfalls and the pacing problem. He lists disrupted organ supply chains, poorly regulated tissue and organ markets, and dual use such as biopreserved pathogens. Governance faces the ““pacing problem,” where emerging science outpaces the development of law, policy, and other governance mechanisms”. He credits Gary Marchant for this. Oversight, though, is “only part” of the landscape, and that is where risk innovation comes in. - Risk innovation restated. He defines it as “a way of bringing an innovation mindset to understanding and navigating the types of risks that are often ignored but have a tendency to bit hard”. It started with the Risk Innovation Nexus for “time and resource-constrained founders”. Its components: - Eighteen orphan risks: “hard to quantify, easy to ignore, yet potentially devastating to an initiative if not considered”. - Risk as “threat to value”: a threat to creating new value or to existing value, and to “aspirational value”. - The enterprise at the centre, surrounded by investors, customers and communities. - Value, not values: values “are important, but the former is more effectively operationalized in policy and decision-making”. - The risk landscape: the ground between where an enterprise is and where it wants to be. It contains quantifiable risks to “human, environmental, and fiscal health” that existing tools can handle, and “risks that are often overlooked because they’re messy, subjective, and hard to deal with”. - Internal versus reciprocal threats: risks that come directly from an organisation’s actions, and risks that arise when it threatens stakeholders’ value, which then comes back to threaten the organisation. - The Risk Innovation Planner: a 30-minute tool (“we almost nailed it”). - The study. Three 90-minute IRB-approved workshops with 17 participants, one each for human health, food systems and biodiversity. Participants mapped stakeholders, over 50 areas of value, and orphan risks, producing a “Risk Innovation Landscape” map. He is candid about the limits: “This plot is dense and subjective”, with no “repeatable” data. - Findings. - Ethics, perception, and government and regulation dominated across all three domains. He expected this; what surprised him was that it came from “experts and practitioners who may otherwise not fully understand how damaging ignoring them can be”. - Privacy, co-opted tech (a useful technology harmed by uses that undermine it in the eyes of investors and users) and bad actors (researchers or companies “that give the technology a bad name”) varied between domains. - Conclusion. The approach raises awareness early and helps innovators navigate “potential roadblocks”. It can help ensure societal benefit “not necessarily by impressing on researchers and developers the need to “do the right thing””. It works instead by showing that “their success is intimately intertwined with how they impact (and threaten what of value to) others around them”.

How firmly. Firm on the framework. Modest on the empirical results. “more work is needed” and the framework “seems likely” to be transferable.

Concepts. Risk innovation. Threat to value (existing and aspirational). Value versus values. Orphan risks (18). Risk landscape. Internal versus reciprocal threats. Co-opted tech. Bad actors. Pacing problem. Responsible and beneficial development. Success-based rather than moral framing of responsibility.

Analogies and technologies. Biotechnology and biomedicine (biopreservation), with a dual-use comparison to pathogens. The framework is explicitly presented as transferable to “other domains of emerging technology”, which is a structural claim. He does not apply it to AI here.

Governance and who decides. He is pluralist. Oversight and law (Marchant) are necessary but insufficient. Enterprises, investors, customers and communities all shape outcomes. Responsibility is framed as enlightened self-interest inside a web of stakeholders.

What he criticises. Naive introduction of beneficial but disruptive technologies. Neglect of messy, subjective risks. Moralising that doesn’t connect to success.

Quotes. - “a way of bringing an innovation mindset to understanding and navigating the types of risks that are often ignored but have a tendency to bit hard” - “risks that are often overlooked because they’re messy, subjective, and hard to deal with” - “the latter are important, but the former is more effectively operationalized in policy and decision-making” (value versus values) - “their success is intimately intertwined with how they impact (and threaten what of value to) others around them”

2024-11-24 · artificial-intelligence-agency-human-amanuensis · “Artificial intelligence, agency, and the emergence of humans as AI amanuenses”#

Provenance. His own prose. The seed was a LinkedIn comment by Mary Burns: “Where’s the satisfaction in being AI’s amanuensis”. He notes (fn 2) that he tried to explore the idea with ChatGPT-4o and Claude 3.5 Sonnet, but the text and the model are his. The diagrams are presumably his.

Argument in his terms. This is a self-described “thought experiment”. - The two-actor model. Human–AI relationships run from AI as tool (the human holds all the power: tasks in, products out), to AI as assistant (more balanced influence, including acting as an agent, but the human stays the “primary agent”), to AI as amanuensis (the human creates, the AI turns the ideas into concrete form and can “stimulate and extend” the human’s creativity). He describes this as symbiosis already visible in text, image and video tools. - Adding a third actor. The two-actor model is unrealistic because “very few people have complete control over what they do”. Everyone is accountable to someone. Adding a primary agent (a boss, or someone with social authority) makes the human a secondary agent. - The flip. If the primary agent works directly with the AI, the AI becomes the idea generator and “co-opts another human” to make the work concrete. That makes the human an amanuensis to AI. He points to the MIT materials-science study, imagines therapeutics discovery, and extends the idea to manufacturing, management, innovation and creative work. He stresses that the model does “not imply intrinsic creative ability within the AI”. The AI “becomes a mediator between the primary agent and the human amanuensis”. - Consequences. - A restructuring of organisations “away from human-centered professional or creative agency, and toward AI-directed and human-executed implementation”. - A small group of people moving into primary-agent roles working mainly with AI systems. - If primary agents are themselves AIs, networks of three-actor building blocks in which agency moves from people to AIs, with humans codifying AI work “into outputs and outcomes that create value for others”. - Normative turn. Even “a small chance” of this future means we should ask whether we want it, “especially if such a shift in role begins to suck the joy out of what we do, even if it does lead to increases in productivity”. - Postscript: human creativity. He is “still not sure that we’re at the point where it can produce completely new ideas rather than reflect what is already known”. That gives him “some hope” that a uniquely human “spark of creativity” remains, “at least for now”, though he wonders how long that will last. He worries more that the model is “so compelling that organizations adopt it at scale — without fully understanding the potential long term consequences to human creativity and innovation”. - A methods observation (fn 2). He found it hard to make the chatbots explore a human-diminishing framing without “fancy prompting footwork”, and says: “I wonder if the guardrails built into these platforms resist explorations that diminish the potential role or centricity of humans”. This is an early note on model design shaping what users can think with.

How firmly. Tentative. He says it “could be completely wrong”. But he calls the three-way relationship “highly plausible” and “in all likelihood, is already beginning to occur”.

Concepts. Human as AI amanuensis. The tool / assistant / amanuensis spectrum. Primary versus secondary agent. The three-actor model as a scalable building block (he flags a nod to Actor Network Theory, “much, much more simplistic”). AI as mediator of agency. Productivity versus joy. AI called “generative” that does not yet generate genuinely new ideas.

On AI. He sees AI as a growing locus of agency in organisational systems rather than as a mind. Its creativity is doubted for now. What is new is its position in chains of delegation and accountability.

On AI risk. The loss of human creative agency and joy at work. Organisational adoption at scale without understanding long-term effects on “human creativity and innovation”. A slow structural risk, not a catastrophe.

On cognition and formation. He hints at de-skilling and the erosion of the creative, idea-generating role that makes work meaningful. Model guardrails may subtly steer what users can explore.

Quotes. - “are we potentially looking at a future where the roles of humans and AIs shift, with AIs becoming the idea generators and humans becoming the equivalent of their amanuenses?” - “a shift away from human-centered professional or creative agency, and toward AI-directed and human-executed implementation” - “I wonder if the guardrails built into these platforms resist explorations that diminish the potential role or centricity of humans”

2025-01-07 · universities-need-to-step-up-their-agi-game · “Universities need to step up their AGI game”#

Provenance. His own prose. It quotes Sam Altman’s new-year blog post Reflections (not his text).

Argument in his terms. - Taking Altman seriously. Altman predicts AI agents will “join the workforce” in 2025. Maynard: “It’s tempting to dismiss this as hyperbole. But given recent advances in using AI models to simulate reasoning and exert control over external systems, I suspect there’s a reasonable chance he might be right.” If AI heads towards Altman’s “Superintelligent tools”, “we urgently need new understanding and thinking on how humanity will successfully navigate the coming advanced AI transition”. - Who can lead? He goes through the options. - AI companies (OpenAI, Anthropic, Google) “dominate the intellectual space” around frontier models and their responsible development. But: “responsible as these companies claim to be (and I think they’re trying hard), they still lack the breadth of vision and understanding that’s necessary to succeed here.” - Governments, “democratically elected ones at least”, have the “societal mandate” to steer AI towards beneficial futures. But “in most cases they lack the imagination, vision, or agility, to achieve what’s needed.” - Research universities could be transformative. But “they also tend to be mired in tradition, convention, and self preservation.” Are they up to the task? “I’m not convinced they are.” He still sees them as the best candidates, because he cannot identify other institutions “that have the mandate and capability”. - Existing efforts. Stanford HAI, Cambridge’s Leverhulme CFI and Oxford’s Centre for the Governance of AI are respected but “barely scratching the surface”. - Against AI and X. Framing initiatives around disciplines, so that ““AI” is simply tagged on to exiting endeavors”, would constrain thinking and turn AI into a funding “buzzword”. - Three intersecting foci, integrated and “domain-agnostic”: - where we live: homes, communities, environment, planet, space; - what we do: discovery, value creation; - who we are: collectives and individual self-understanding. - What it would take. “billions of dollars”. Funding without stifling strings, “more usually associated with philanthropists and foundations rather than government agencies”. Breaking institutional constraints and conventional metrics. Mould-breaking scholars who work across domains. - The sting. If someone finds a better approach that doesn’t rely on universities, “it’s probably time for these hallowed bastions of academia to ask what value they actually bring to the table”.

How firmly. The urgency is firm. The institutional answer is hedged and provocative.

Concepts. Advanced AI transition(s). Leaders versus bystanders. The three foci (where we live / what we do / who we are). Domain-agnostic, integrated scholarship versus AI and X. Human flourishing “in an age of AI”. Universities as institutions for “the creation of public value”.

On AI. He treats AGI and agentic AI trajectories as live possibilities and cites simulated reasoning and control of external systems. The framing (“AI-dominated future”, “poised to transform every aspect of our lives”) is transformational.

On AI companies and leaders. Fairly generous to their intentions (“trying hard”). Critical of their narrowness and of their control of the intellectual agenda.

On governance. Democratic mandate lies with governments, but capability is lacking. He wants an independent, public-value-oriented third force for knowledge and “thought leadership” rather than regulation as such.

Links. A direct AI application of his earlier call for new institutions and pilots for advanced technology transitions (B14) and for public-university-based futures thinking (B15 FHI post).

Quotes. - “responsible as these companies claim to be (and I think they’re trying hard), they still lack the breadth of vision and understanding that’s necessary to succeed here” - “in most cases they lack the imagination, vision, or agility, to achieve what’s needed” (governments) - “they also tend to be mired in tradition, convention, and self preservation” (universities) - “we urgently need new understanding and thinking on how humanity will successfully navigate the coming advanced AI transition”

2024-12-13 · are-educators-falling-behind-the-ai-curve · “Are educators falling behind the AI curve?”#

Provenance. His own prose. It quotes Sundar Pichai and summarises Ethan Mollick’s “15 Times to use AI, and 5 Not to”. The s-curve figures are his.

Argument in his terms. - The lag. Colleagues’ ideas of what AI does and how students use it are out of date. The lag between what educators think and what is happening “is widening at a frightening rate”, with effects from individual teaching to institutional integrity policies. - The three s-curves model. 1. Capability curve. “there are indications that large language models are following a classical tech innovation “s-curve””, now approaching a plateau (Pichai: “the low-hanging fruit is gone”). 2. Utilization curve. This lags capability. Products that create “tangible value” come later. Where both curves are steep, the gap between potential and use is large: “We’re at a point where progress at the cutting edge of AI is slowing down, while advances in how AI is being used are speeding up.” 3. Perception curve. This lags both. People’s “mental models” were fixed by ChatGPT in 2022: a prompt-and-response tool whose outputs were “seductive, persuasive, and — in the case of students using the platform to complete assignments — deceptive”, with hallucinations and “tells”. - What AI now is. “the generative AI of today is much closer to having a smart, engaging, and supremely patient artificial person integrated into everything you do.” It is embedded in Siri and Alexa, in voice personas and Perplexity, in writing and coding, and even in “just thinking about and noodling with new ideas”. It is more accurate, holds longer conversations, can emulate reasoning, and is so deeply integrated that “it’s sometimes hard not to use them (or even to know when you are)”. - Students are ahead. They learn by hands-on experience faster than instructors. Outdated policies misjudge “what students need to know” and penalise them “if they don’t confirm to outdated expectations”. - Remedies. - AI literacy, but focused “on how to think about the development and use of AI within society, rather than the nuts and bolts of what the technology is capable of”: responsible and irresponsible development and use, the AI–society intersection, “critical failure modes that are likely to persist across multiple generations of AI”, and navigating shifting “limitations and affordances”. - Agile, technology-independent policies, not “black and white” ones. Integrity policies built on 2022 ChatGPT will be “highly brittle”. - He points to “a wealth of insights that I suspect could be drawn from work around agile governance”. - He endorses Mollick’s approach, including not using AI “when you need to learn” or “when the effort’s the point” (Mollick’s items, which he endorses).

How firmly. Confident on the lag diagnosis. Hedged on the plateau (“some will disagree”). Uncertain on solutions (“I’m not sure what these might look like”).

Concepts. Capability / utilization / perception s-curves. Mental models stuck in 2022. Embedded or ambient AI. AI literacy as ways of thinking. Persistent failure modes. Agile, technology-agnostic governance of AI in education. Responsible and beneficial use.

On AI. A quasi-person: a “supremely patient artificial person” woven into daily life and thinking. It is now ambient, not a discrete app.

On cognition and formation. Implicit. AI is already part of how students think and learn. He notes, through Mollick, that effort and learning can be undermined. His main worry here is institutional lag rather than cognitive harm.

Governance. Agile governance transposed to institutional education policy.

Quotes. - “We’re at a point where progress at the cutting edge of AI is slowing down, while advances in how AI is being used are speeding up.” - “the generative AI of today is much closer to having a smart, engaging, and supremely patient artificial person integrated into everything you do” - “being aware of critical failure modes that are likely to persist across multiple generations of AI”

2024-11-10 · is-ai-poised-to-suck-the-soul-out-of-science · “Is AI poised to suck the soul out of science?”#

Provenance. His own prose. There is a long block quote from Mark Daley’s Substack and a quote from Aidan Toner-Rodgers’s paper, neither of which is his. A later update (19 May 2025, his) reports serious doubts over the paper’s integrity and MIT’s call for it to be withdrawn. He keeps the post up but asks readers to “read with caution”.

Argument in his terms. - Context. He has written enthusiastically about AI accelerating discovery: his “supercharging research” piece and the WEF Top Ten Emerging Technologies 2024 write-up on AI for scientific discovery. So he was “quite shocked” by the study as Daley presents it: 44% more materials discovered, 39% more patents, but 82% of scientists less satisfied, with AI taking over idea generation. - The soul of science. Daley’s commentary “disturbed me more than I was expecting”: “the soul of science lies in the delight and wonder of exploring the unknown rather than just being a cog in a knowledge production line”. - Societal good, redefined. Science must be justified by societal good, “to a point”. But “how “societal good” is defined has to be broader than short term financial or material gain”. It must include long horizons (“tens or hundreds of years”) and “what scientists get out of the deal”. - Curiosity as fuel. Drawing on “my career-long experience as a scientist”, he says wonder, freedom to ask “how” and “what if”, and curiosity are the fuel of discovery, often serendipitous. Remove it and scientists become unhappy. - Being human. “science is also an expression of who we are”. Take away curiosity and wonder, and “science simply becomes a utilitarian tool to support a utilitarian world heading for a utilitarian future”. He adds: “a little bit of all of us dies in the process.” - A self-defeating loop. A soulless science erodes the societal good it is meant to serve, because who would become a scientist? - Hopeful close. There are probably ways of using AI that “spark curiosity and increase the joy and wonder”. This will need “new thinking” and “intentional steps” to “ensure the “rightful place” of AI in science”. The phrase echoes Obama and the CSPO book series.

How firmly. Emotionally firm on values. Tentative on the evidence, and more so after the retraction note.

Concepts. The soul of science (joy, wonder, curiosity). A broadened definition of societal good. Scientists’ own return on investment. The utilitarian knowledge production line. The “rightful place” of AI in science.

On AI risk. A formation and meaning risk: AI displacing the most intellectually rewarding tasks (idea generation) and turning experts into “highly educated lab technicians” (Daley’s phrase). This is not a safety risk but a threat to value in the broad sense.

Change of view. Not a reversal. It qualifies his earlier enthusiasm for AI-accelerated discovery. The May 2025 note shows him publicly updating his evidence base.

Quotes. - “the soul of science lies in the delight and wonder of exploring the unknown rather than just being a cog in a knowledge production line” - “how “societal good” is defined has to be broader than short term financial or material gain” - “science simply becomes a utilitarian tool to support a utilitarian world heading for a utilitarian future”

MEDIUM#

2024-11-17 · navigating-the-ethical-dilemmas-of-brain-computer-interfaces · “Navigating the Ethical Dilemmas of Human-Enhancing Brain-Computer Interfaces”#

Provenance. Mixed. Much of the post summarises and quotes Emma Gordon and Anil Seth’s PLOS Biology essay, including a long quoted “free reign” passage and quotes from Sandel via them; none of that is his. His own contributions are his framing, his evaluative asides, his extensions of their six questions, and his footnotes. He discloses membership of the CIFAR Research Council.

Argument in his terms (his parts only). - Continuity. In 2019 he and Marissa Scragg wrote one of the first responses on the ethics of Neuralink’s BCIs. Five years on, questions “we began asking in 2019 have taken on much greater urgency”. - Method: plausibility over hyperbole. He praises grounding ethics in “plausible challenges rather than hyperbole”. He likes their move of letting imagination run and then reining it in. He calls this “a technique that I’ve used in the past”, useful for “closing the shutters on hyperbolic speculation”. He calls it “a critically important step toward ensuring ethical questions and constraints are grounded in plausibility rather than hyperbole”. Their line that imagination should be “reined in by what is … practical, feasible, and ethical” is one “every entrepreneur, founder, and student studying innovation, should have stamped indelibly in their thinking” (fn 5). - Science before engineering. He endorses their space-flight analogy: BCIs are not “rocket science” because the underlying science is lacking. Brute-force trial and error is ethically questionable. AI might “soften” the science gap. He reports this possibility without endorsing it. - His extensions of the six ethical questions. - Privacy. Neural data as “high-value training data for artificial intelligence models”. He pushes further than the authors: users may be incentivised to “share” neural data through free or reduced-cost subscriptions. - Inequality. It seems likely that companies will find “creative ways of … extracting value from users who can not otherwise afford the technology”. - Mental monoculture. New to him. He adds that it “hints at a more sinister manipulation of “group think” by dominant producers, controlling governments, or even AI systems integrated into increasingly sophisticated eBCIs”. - Inauthenticity. This is not unique to eBCIs. It “applies to pretty much any technology” that affects our sense of “true self”. Direct brain manipulation raises it “to a new level”. - Cheapened achievements. He takes both sides. If unaided human achievement defines being human, eBCIs could be “an existential threat to who we are”. But “we’ve been using artificial means to enhance our minds in order to achieve stuff for millennia”. He also asks whether identity lies in what we achieve as a society. - Autonomy. Misreading of signals, AI acting as “an electronic conscience or set of guard rails that decide what we should or should not do”, malicious overrides, and BCIs writing thoughts in, so that “what we think of as our ideas are, in fact, not”. - Hyperagency. He is sceptical that users will spiral under Sandel-style excess responsibility. His reason: “Watching how many people seem to be quite comfortable with behaving irresponsibly in the face of current technological capabilities”. He allows that he “may be wrong”. - The skull as boundary. He reports their view that the skull preserves “the idea of autonomy”, and endorses the conclusion that we ignore these questions “at our peril”. - Musk (fn 1). The 2019 Neuralink paper lists Musk as first author: “this ethically dubious choice of authorship”.

Concepts. Grounding ethics in plausibility. Disciplined imagination. Neural data as AI training data. Incentivised data sharing. Mental monoculture and manipulated group think. Authenticity as a general technology question. Enhancement as a millennia-old continuity. Hyperagency (Sandel’s, via the authors).

Links to AI. AI appears as a data sink for neural data, as a possible manipulator of collective thought via eBCIs, and as an embedded “electronic conscience”. These are cognition, manipulation and autonomy concerns that parallel his AI-persuasion work.

Quotes. - “ensuring ethical questions and constraints are grounded in plausibility rather than hyperbole” - “It also hints at a more sinister manipulation of “group think” by dominant producers, controlling governments, or even AI systems integrated into increasingly sophisticated eBCIs.” - “we’ve been using artificial means to enhance our minds in order to achieve stuff for millennia”

2025-01-12 · chatgpt-faq-education · “Frequently Asked Questions on Using ChatGPT in the Classroom”#

Provenance. The post is his own prose. The FAQ it links to (Google Doc/PDF, not in the post) was produced with ChatGPT-4o, and he says “ChatGPT did the bulk of the work”. His role was curating, editing and asking questions. The original 2023 FAQ was also compiled with ChatGPT. The FAQ document is therefore not evidence of his thinking. His account of the process is.

Argument in his terms. - Why he updated it. ASU guidelines linked to his outdated 2023 FAQ. He had put off updating it because of the workload, which was “me in pre-ChatGPT thinking mode”. - The process. Iterative and “generative”: upload the old items, get revisions, have ChatGPT review the whole set, select and refine. It took about four hours of clock time, or 1.5–2 hours of actual work. - The committee comparison. He estimates a university committee would have spent months and over 200 person-hours on something “that might or might not be fit for purpose”. He admits this is “a tad on the disingenuous side”. - Main takeaway. “Generative AI is limited and flawed. But it’s also a game changer in a growing number of situations when used appropriately.” Failing to acknowledge this, especially in education, will produce a gap between those learning to use AI well and those who are not. This is a capability-based inequality. - Expertise. He describes the process as “ChatGPT substantially amplifying and extending my existing expertise”. “Without me having a good grasp of what I wanted and how to ensure it was fit for purpose”, the output would have been worse. - Language of collaboration (fn 1). He deliberately uses “we”: “I’m not sure we have the appropriate language yet to describe working with machines that are fundamentally different from passive devices.”

Concepts. AI as amplifier of existing expertise, not a substitute. The “we” of human–AI co-production. Machines that are not passive devices. The AI capability gap in learning. Institutional slowness (committees) versus AI speed.

On AI. A new kind of collaborator that needs new language. Flawed but transformative when paired with expertise.

AI in scholarship and writing. Openly publishing, under a CC licence, material largely drafted by AI, with a description of the division of labour. This continues his transparent practice of experimenting with AI as a co-producer.

Quotes. - “Generative AI is limited and flawed. But it’s also a game changer in a growing number of situations when used appropriately.” - “I’m not sure we have the appropriate language yet to describe working with machines that are fundamentally different from passive devices” - “ChatGPT substantially amplifying and extending my existing expertise”

2024-12-01 · geoengineering-aerosol-monitoring-john-aitken · “Geoengineering, early warnings, and a dash of Victorian science”#

Provenance. His own prose. It includes a quoted passage from John Aitken (1890, not his) and two quoted passages from his own 2015 Nature Nanotechnology article, which is his prose.

Argument in his terms. - Geoengineering. He opens with a NYT story on NOAA’s stratospheric aerosol monitoring network, an “early warning system” for detecting covert or unilateral solar geoengineering. The “ethics and politics of such geoengineering approaches are far from clear — especially when the global consequences of unilateral actions are highly uncertain”. Monitoring therefore “makes sense”. He notes (fn 1) that he has written about geoengineering and responsible innovation since 2009 (ocean iron seeding, “geoethics”). - Aerosol history. The post is mainly the aerosol physicist in him tracing present techniques back to Aitken’s 1880s–1890 dust counter. That counter led to condensation particle counters, which are “the basis for routine workplace nanoparticle concentration measurements”. This is his nanomaterials-exposure background. - Aitken on climate. Aitken speculated about particles, temperature and human activity (coal, population). His mechanism was the inverse of the one stratospheric geoengineering relies on, but he recognised a human influence on climate early. - General lesson. Innovation “often have their roots in research that goes back decades — and often more”. Quoting his 2015 self: “seemingly novel challenges don’t always demand novel solutions, and sometimes, the key to moving forward safely, is to look back at what’s already known.” He also objects to the attitude of a student who asked “can we trust papers more than a few years old?”

Analogies. Literal technical lineage (Aitken, then CPCs, then nanoparticle exposure measurement). Conceptual: past knowledge as a resource for safe innovation.

Governance. Unilateral geoengineering needs monitoring and detection. The implication is that monitoring infrastructure underpins governance of technologies with global effects.

Quotes. - “the ethics and politics of such geoengineering approaches are far from clear — especially when the global consequences of unilateral actions are highly uncertain” - “seemingly novel challenges don’t always demand novel solutions, and sometimes, the key to moving forward safely, is to look back at what’s already known” (his 2015 text, re-quoted)

2024-12-29 · fantasy-top-ten-lists-2025 · “Someone needs to write these Fantasy Top Ten Tech lists”#

Provenance. His own prose. The header video was generated with Sora. The substance is in his footnotes.

Argument in his terms. A playful “top ten list of top ten lists”, whose footnotes carry real positions: - The pencil-and-paper test (fn 4). A “tongue in cheek” test he uses on hyped edtech. It asks whether a technology solves problems that are unsolvable in other ways, or whether “we’re simply trying to manufacture problems that we claim emerging technologies can solve — thus retroactively justifying their development and use”. - What AI won’t change (fn 5). AI “are transforming the world”, but “it’s always good to stay grounded in what a transformative technology is not doing”. - Joy (fn 7). Joy is “much under-appreciated” in thinking about technology: “many advances have a nasty habit of sucking the joy out of what we do without us realizing it!” This links to the soul-of-science and amanuensis posts. - Hype (fn 9). “Hyperbole is the primary currency in an attention economy.” - Wow moments forgotten (fn 8). People are “incredibly immune” to transformative knowledge. He sees this partly as a survival instinct that is “part of what makes us human”. - Tech bros and governance (fn 11). “a bunch of tech bros forget everything they ever knew about the Dunning-Kruger effect when it comes to governance and policy”. This is written in the post-election context of tech leaders entering government. - Being human (fn 12). Whether technology is changing what it means to be human is “open to debate”. But with novel AI, humanoid robots, BCIs, new understandings of consciousness and bioengineering, “it’s getting harder to hold on to what we’ve thought of as the bedrock of being human as if it’s an immutable truth”. - List item 1. He asks for book lists without “Musk”, “Bezos”, “Altman” or “Gates”. A dig at tech-celebrity dominance of the conversation.

Concepts. Pencil-and-paper test. Manufactured problems and retroactive justification. Joy as a neglected value. Hyperbole as attention currency. Tech-bro overconfidence in governance. The mutability of “being human”.

Quotes. - “many advances have a nasty habit of sucking the joy out of what we do without us realizing it!” - “a bunch of tech bros forget everything they ever knew about the Dunning-Kruger effect when it comes to governance and policy” - “it’s getting harder to hold on to what we’ve thought of as the bedrock of being human as if it’s an immutable truth”

2024-12-20 · sora-has-a-bias-problem · “Sora has a bias problem”#

Provenance. His own prose and experiment. It quotes the Sora System Card (OpenAI’s text, not his).

Argument in his terms. He asked Sora sixteen times for “an academic giving a lecture”. He got sixteen men: fourteen white, two Black, thirteen bearded, all in button-down shirts. Early image generators showed occupational stereotypes, and OpenAI and Midjourney have “made huge strides in correcting these biases”. Sora “hasn’t quite caught up”. He is “sure that OpenAI will plug this bias gap … pretty soon. But I’d expected better from them.” He wonders what other biases are embedded. His jab at staffing: “If only these companies were employing more people who could help ensure the technology’s responsible development.” Then: “Sigh …”

On companies. He credits progress but expects more. He hints that AI firms under-invest in responsible-development expertise.

Risk framing. Representational bias as a persistent responsible-development failure.

Quotes. - “Clearly we still have a long way to go in de-biasing generative AI.” - “If only these companies were employing more people who could help ensure the technology’s responsible development.”

2024-11-06 · ai-in-a-world-of-trump · “AI in a world of Trump” (Modem Futura promo; his written framing only)#

Provenance. A podcast promo. Only the framing paragraphs, his own prose, are noted here. The episode (on Turing’s “Can machines think?”) was not reviewed.

Argument in his terms. Written the morning after Trump’s election: - The likely policy shift. Signs of reduced government oversight, a push for rapid growth, and “embracing a more “permissionless” approach to innovation”, especially given Musk’s relationship with Trump. - Likely effect. An emphasis on “US-centric short term gains” promising long-term rewards, “unhindered by overly restrictive government regulation”. - Who governs. Responsible and beneficial AI “will depend less on government oversight and more on a tapestry of soft governance mechanisms that rely increasingly on developers and their key stakeholders — including consumers”. He presents this as the new reality rather than endorsing or attacking it. - The goal. AI “that improves lives rather than makes them worse — whoever’s at the political helm”.

Concepts. Permissionless innovation (in scare quotes). Soft governance. Developers and consumers as governance actors.

Quotes. - “a tapestry of soft governance mechanisms that rely increasingly on developers and their key stakeholders — including consumers”

LOW / NONE#

• 2024-12-15 · advanced-ai-challenges-opportunities-native-american-communities (low). Mostly guest reflections by Al Kuslikis (“Regenerative AI”, Indigenous AI), Sean Dudley (Midjourney’s biased renderings of Tuba City), and Leonard Bruce (worries about “tech oligarchs”, going local, new stories of AI). None of that is his prose. His short intro calls the image bias “a stark reminder of the biases that still plague generative AI platforms and disadvantage minority groups and communities” and frames the collection as showing “the need to be more inclusive and imaginative”. Relevant to inequality and justice through his editorial choice to publish these voices.
• 2024-12-08 · guide-to-thinking-about-the-future (low). A short promo for his 2020 book Future Rising. He admits of the “architects of the future” metaphor, “I’m still not entirely sure how useful this metaphor is.”
• 2025-01-05 · five-voices-five-pieces (low). Recommends Substacks by Mark Daley, Jamey Wetmore, Andy Revkin, Ethan Mollick and Melanie Mitchell. His one substantive aside, on Mitchell’s o3 post: models like o3 are “pushing far beyond critiques that large language model-based AIs are simply “stochastic parrots””, yet “we still don’t know what this means in terms of human-like reasoning in machines”.
• 2024-11-12 · waymo-we-go (none). Modem Futura episode notes on autonomous driving and Sean Leahy’s first Waymo ride. Set aside per user instruction.
• 2024-12-22 · why-modem-futura-is-more-than-just-another-tech-podcast (none). A reflection on the podcast’s tenth episode. Set aside per user instruction.
• 2024-12-31 · modem-futura-beyond-the-horizon-space-technology (none). Modem Futura end-of-year episode with space researchers Caity Roe and Joe O’Rourke. Set aside per user instruction.
• 2025-01-07 · a-conversation-with-cady-coleman (none). Modem Futura episode with former astronaut Cady Coleman. Set aside per user instruction.