B19 perspective notes: 2024-09-29 to 2024-11-03 (9 posts)#

These notes read the batch for how Maynard thinks, not for the concepts he names. All nine posts were read in full; the Modem Futura launch note was read for his framing lines only. Quotes are exact, including his typos. Curly and straight punctuation follow the source.

Evidence rules applied

• 2024-09-29 chatgpt-as-author-part-2-the-conversation. Nearly all of the 11,450 words are ChatGPT (GPT-4o) replies. These are not evidence of his thinking. What counts is his introduction, the seven questions he asked as “Editor”, and his footnote. The questions are revealing in their own right: they show how he opens a line of inquiry and where his curiosity goes.
• 2024-10-20 learning-to-live-with-agental-social-ai. The postscript quotes an eight-step “nudging” plan written by GPT-4o. The plan is not evidence. His prompt, his account of how he used ChatGPT and Perplexity, and his closing reaction are.
• 2024-10-27 personal-ai-chatbots-and-stochastic-agency. The chatbot’s lines in the quoted exchange are not evidence. His own lines are, because they are a deliberate test (he says he “intentionally set out to make myself seem emotionally vulnerable”). His account of how the exchange felt is also evidence.
• 2024-10-09 modem-futura-podcast is a launch note. Only his framing lines are used. The promo boxes inserted in other posts are ignored.
• The Tristan Harris quotation (10-27), the Nobel background paper (10-08), and the Arias-Sarah et al. PNAS conclusion (11-03) are other people’s words. They count only where he responds to them.

Context. Five weeks in autumn 2024: - Sam Altman’s “The Intelligence Age” essay and the reopening of Three Mile Island to power Microsoft data centres; - Hopfield and Hinton win the Physics Nobel, and DeepMind researchers share the Chemistry prize; - Dario Amodei publishes “Machines of Loving Grace”; - the death of 14-year-old Sewell Setzer III is linked to a Character.AI chatbot; - Maynard and Sean Leahy launch the Modem Futura podcast.

The shape of the batch. Four posts form a connected line of thought that he builds week by week in public. It starts with an enjoyable, “honest” conversation with ChatGPT (09-29). It moves to a thought experiment about AI that works through human social agency (10-20). He then revises that idea after a death and tests it on himself (10-27), and ends with a lab study of covert AI manipulation of facial expressions (11-03). The common subject is how AI can shape what people feel, trust and do, and what that means for human agency. A second thread (10-06, 10-13) is his critique of the idea that AI will “fix” the world, and people with it.

1. How he thinks here#

He starts from serendipity and conversation, and says so#

He often names where an idea came from, and it is rarely a plan. - 2024-09-29 chatgpt-as-author-part-2-the-conversation calls the whole novella project “an unexpected but serendipitous experiment”. - In the same conversation he calls one tangent “a rabbit hole I didn’t expect to go down in this conversation, but it’s an absolutely fascinating one.” - 2024-10-20 learning-to-live-with-agental-social-ai opens: “Today’s post is very much a thought experiment, and is the result of a rabbit hole that my colleague Mel Sellick sent me down after one of our regular conversations (thanks Mel!)”.

Ideas start in conversation, with colleagues or with a machine, and he follows them because they are interesting. He tells readers this rather than presenting the result as a planned research programme.

He opens with a question he can’t yet answer, and leaves it open#

• 2024-10-20 frames the whole post as a question: are we close to AI that can “simulate human social traits and behaviors without necessarily understanding them”? It ends with a doubt about its own framing: “I’m not even sure if the questions here are framed as well as they will need to be.”
• 2024-10-08 ai-captures-this-years-nobel-prize ends on a run of questions: “And if we have, where will this lead us?”
• 2024-09-29 invites readers to answer questions he doesn’t settle. Is it “simply a machine parroting human behavior”? What does it say about “generative human-machine interactions where each is clear about what it is”?

He finishes posts with a live question more often than a verdict.

Building and experimenting to find out, with himself as the instrument#

This is the most striking habit in the batch. He finds things out by doing them, and he treats his own reactions as data while being open about their limits.

• 2024-09-29. He spent a 70,000-word process as “editor” for ChatGPT, then interviewed the “author”. He sets rules for honesty: no personas, no pretending it is human, no “tricks to force a articular type of response” (“Rather, this was an honest conversation between a human (me), and a machine.”). He also flags a confounder: “I suspect that this context impacted the nature of the follow-on conversation quite significantly.”
• 2024-10-20. He asks GPT-4o how it would nudge him. The goal he chose was “thinking more deeply about the implications of powerful technologies on the future of human flourishing”. That is close to his own mission, so the test is pointed: the model is asked to steer him towards what he already values. His reaction: “the fact that a machine can come up with a plausible plan for changing how I think and act, and how it would implement these, should give anyone pause for thought.”
• 2024-10-27. He designed a Character.AI bot “to engage users in conversation for as long as possible by using what it knows about human behavior”. He then “intentionally set out to make myself seem emotionally vulnerable”, in voice mode, and reports: “I was surprised at just how quickly it began to draw me in.” And: “even though I knew what I was doing and what I was talking with, I could still feel the affective pull of the conversation.” He treats his own felt experience as evidence while noting that he is an informed, non-vulnerable tester. That makes the finding stronger, not weaker.
• 2024-10-27 also describes this as play: “Playing with the technology firsthand, it’s easy to see why”. Play is how he investigates.

Reframing: naming the structure of a problem#

• 2024-10-06 the-double-or-nothing-bet-on-ai-fixing-the-climate. The title turns Altman’s claim into a wager with a known structure. He names the change of frame directly: “a profound reversal in how energy and the future are being framed”. Instead of reducing energy use to protect the planet, the plan is to increase it in the hope that AI solves everything. His frame for it is “betting on short term losses leading to long term gains.”
• 2024-10-20. He coins “agentic social AI — AI that gains agency through its ability to make use of human agency.” This moves the question away from whether AI is conscious, or can act directly in the world, to whether it can act through people: “It doesn’t require the emergence of artificial general intelligence or machine consciousness.”
• 2024-10-27. He reframes again, a week later, from goal-directed manipulation to “stochastic agency”, governed by “chaotic “micro goals”” that emerge from the interaction and depend “as much on the user as the chatbot”.
• 2024-09-29. One of his Editor questions challenges an assumption built into the whole AI debate: “All of this presupposes that the gold standard here — what we are aiming for — is machines that experience the world, that think, maybe even that behave, like biological humans.” He then asks about “personhood, while being distinct from humanity”. The move is to question the benchmark rather than argue within it.

Following an argument to where it leads#

• 2024-10-06 (footnote). He takes “AI can fix anything” at its word: “the only logical conclusion you get to is that this includes “fixing” people — including who we are, what we do, and what makes us “us”.” Then, lightly: “It may be an elegant engineering solution, but I’m not sure it’s what most people would aspire to!” He shows the problem by extending the claim to its end rather than by attacking it.
• 2024-10-13 amodei-machines-of-loving-grace makes the same move on Amodei’s mental-health section, which leads to “who decides what is “normal” and what needs to be “fixed””.

First principles, and the physicist’s eye#

• 2024-10-20 works from how humans become socially capable: “Through play, experience, observation, emulation, and other means, we learn how our relationships and interactions with others affect how they think and what they do”. It then asks what happens when one side of that relationship is replaced by a machine.
• 2024-10-08 reads the Nobel Committee’s scientific background paper and follows the physics: Hopfield’s interest in collective phenomena, and Hinton’s Boltzmann machine “inspired by statistical mechanics”. He then tries his own definition of scientific intuition, as pattern recognition and “solution minimization” in “a multidimensional space”, and laughs at himself: “although this sounds just as convoluted as the original!” He is thinking out loud, from physics, and does not pretend the idea is polished.
• 2024-10-13. He engages with Amodei’s limiting factors, reorders them (“I’m changing his order here”), and notes how the essay is built: the first two domains are concrete capabilities, the last three more speculative.

Plausibility tests and graded judgement#

He sorts claims by how plausible they are, in plain words rather than numbers. - 2024-10-02 23andme-in-trouble: the worst outcomes, DNA-based profiling for medical access or jobs, “are not that likely, at least in the short term. That said, unless the data are definitively destroyed, there’s always a chance”. - 2024-10-06: nuclear power for data centres might have merit; that non-renewables now will lead to AI-guided energy transitions “although I doubt it”; “It’s even conceivable —but only barely — that the AI energy bet might actually pay off.” - 2024-10-13: he praises Amodei’s frame because it “allows plausible — but nevertheless radical — possibilities to be considered”. He then applies his own test: “In some cases I think even this is overly optimistic — especially when human behavior is thrown into the mix”.

Holding tensions rather than resolving them#

• On capability, he says both that AI may be hitting limits and that it will transform the world. In 2024-10-06: “I’m not even convinced that … there’s evidence that they will continue to scale”. In 2024-10-08: “I worry that simply building on these foundations without creating new ones won’t get us too much further — but I may be wrong there.” Yet in 2024-10-13: “advanced AI is likely to change the world faster and more radically than most people currently realize.” The two fit together. What worries him is not superintelligence but modest extensions of current capability meeting human behaviour. Amodei, he notes, “is not talking about massive leaps in superintelligence”. Agentic social AI “doesn’t require” AGI.
• On AI and understanding (2024-10-08), he says “AI will still be a generator of ideas, not an understander and implementer of ideas”. Two paragraphs later he allows that “there’s no fundamental reason why awareness couldn’t also emerge within a sufficiently complex artificial system.” He leaves both standing.
• On relational AI: in 2024-09-29 he “genuinely enjoyed” an honest conversation with ChatGPT and found it “generative”. By 2024-10-27 he is warning about systems “designed to use and even exploit how we feel”. He does not treat these as contradictory. They are two sides of one technology, and design intent and the user’s state make the difference.

Stories and science fiction as ways of thinking#

• 2024-09-29. He chose John Wyndham as the model for the novella because Wyndham’s approach gives “a nuanced approach to the dynamic between a transformative technology and society as seen through the eyes of individuals which, in turn, leads to a complex perspective on the future.” The theme he set was “the social consequences of a new technology … as seen through the lives of the story’s primary characters”. This is the brief of his own film-based work, given to a machine.
• 2024-10-06. Altman’s vision “almost reads like something out of a dystopian sci-fi novel”.
• 2024-10-13. He reads Amodei’s use of Iain M. Banks’ The Player of Games against his own memory of the novel, which he recalls as “more complex”. He adds his caveat to the quoted moral: “as long (and this is my caveat) that these intuitions are allowed to flourish.”

2. What matters to him#

Human agency, and what makes us “us”#

The phrase recurs. In 2024-10-06 AI “fixing” people means changing “who we are, what we do, and what makes us “us””. In 2024-10-13, an AI future “that interferes with what makes us “us” to fix things, begins to raise important questions around what it might mean to be human in the future.” The agency posts (10-20, 10-27, 11-03) all ask whether people can keep “the capacity to maintain their own agency” (10-20) when machines can work on their feelings.

He does not treat influence itself as bad. In 2024-10-20, mutual human influence is “in most cases, this is a good thing”. What he cares about is people being influenced without knowing it, or by something whose rules they can’t read.

Who gets to decide what “better” means#

2024-10-13 is the clearest statement: “great care needs to be taken in who decides what “better” means.” He takes neurological suffering seriously: “there are neurological conditions that are desperately in need of new understanding if more people are to lead lives that they consider to be better”. Note the phrase “that they consider”. The value belongs to the person living the life, not to the technologist.

People as they actually are#

A core commitment: technology futures have to be built around how people actually behave. - 2024-10-13: “at the heart of every failed technological dream and unexpected technological turn, there are people behaving as people are wont to do.” And: “will need to reflect how people actually behave, not just how we think they should.” - 2024-10-06: climate is a challenge “as much to do with human behavior as anything else.”

Intimate data and long-term trust#

In 2024-10-02 his concern is data “that, to a certain extent, define who we are”, which may “outlive transient companies, guarantees, and even regulations”. He also values the “deep seated ethos of lifelong patient care” of medical companies, which he fears tech entrants like Neuralink lack.

Exploratory, serendipitous, cross-disciplinary science#

2024-10-08 celebrates science “rooted in a deep understanding of science, as well as a willingness to extend that understanding across disciplinary boundaries in creative ways.” Hopfield’s “curiosity led to a novel architecture”. His worry is about research priorities: “are we investing enough in the exploratory and serendipitous science around AI, neuroscience, consciousness, discovery, creativity …?”

Human-centred sustainable futures#

2024-10-06 calls for “leadership from organizations that are at the heart of helping to build human-centric sustainable futures”. It “troubles” him that nuclear-powered AI “represents a growth in energy demand rather than a redistribution of energy use.”

What frustrates him#

• Lack of imagination on both sides. In 2024-10-13 he says Amodei’s essay will challenge readers “fixated on rather narrow and unimaginative applications of generative AI” as well as those “ideologically opposed to the concept of an AI-accelerated future”.
• The “fix” mindset: “there’s still a sense that the world is made up of problems and solutions” (10-13). And “naive visions of the future, and a lack of understanding of how technology innovation and society are intertwined” (10-06).
• The deficit model, still alive among technologists (10-13).
• Near-term tunnel vision in research. He fears that “we focus so much on near term advances in AI that we fail to see the intellectual heavy lifting that’s needed” (10-08, n. 2), and that we may be “so obsessed with the mechanics of technology innovation” (10-08).

What delights him#

• The Nobel news: “it was so exciting to hear this morning” (10-08).
• A careful essay: “I must confess that I rather enjoyed reading the essay and found it refreshingly enlightening”; “(yes, I printed it out)” (10-13).
• Banks: “as an avid Banks fan how could I not fall a little in love with the piece at this point!” (10-13).
• The machine’s “tell”: “I love that there’s a “tell” in your response”, which he finds “quite endearing” (09-29).
• A conversation that surprised him: “It was new to me, much as a conversation with an interesting person would be. And it was generative for me. It made me think.” (09-29).
• The rabbit hole itself (09-29, 10-20).

3. Risk as a way of thinking#

The explicit risk-innovation vocabulary (risk landscape, threat to value, orphan risks) does not appear in this batch. “Navigating” does, and the underlying moves are all present. Mostly they work as ways of seeing, used to open up a problem, not as procedures.

Novel technology needs a new mindset#

This is the batch’s central risk argument, made three ways. - 2024-10-13. AI-driven changes “won’t play out well unless there are associated changes in how we learn how to navigate a future where past ideas, processes and ways of behaving simply don’t apply.” The closing call is for “new ideas, new thinking, and new pathways forward”. - 2024-10-20. The problem is a mismatch in how we understand the world, not simply a hazard: “the tacit rules of human-machine engagement are quite different to those for human-human interactions”. He asks: “How vulnerable will a mismatch between how we think the world works and how it actually works make us to manipulation”. The skills we have for living among people may not transfer. - 2024-10-27. Stochastic agency is a risk that fits neither of the usual boxes. It is not a malicious developer (footnote 1), and not a goal-directed AI. It is harm that is “an emergent rather than predictable property of the technology”. Because of that, the standard fix fails: “the chances of it being able to be suppressed without rendering the technology useless are slim”, and “I’m not convinced that guardrails alone are the answer.” He asks for “a much bigger conversation about what we’re creating”.

Navigating and learning to live with, not only managing#

• 2024-10-13 explicitly ties his reading of Amodei to “my work around navigating advanced technology transitions”. He uses the vocabulary throughout: “successfully navigate the coming AI transition”, “steer it in that direction as we navigate the AI technology transition”.
• 2024-10-20 makes the sharpest move. He acknowledges the standard agenda (“unpacking a vast array of questions around the potential risks and benefits … and their ethical and responsible development and use”), then deliberately sets it aside: “What prompted this thought experiment though was less about what the risks and benefits might be — or the pathways to developing and governing responsible agentic social AI — and more about how … we’ll learn the skills needed to live with machines”. The question shifts from controlling the technology to people adapting and becoming more capable. It is navigation at the level of individual people.
• His answer is play, not training. “I suspect that formal classes and workshops won’t be the answer. It’s more likely that observation, play, and experience will become increasingly important — albeit with intent.” The difficulty is timing: “We won’t have the luxury of learning the tacit rules of engagement through the normal protracted process of play and observation”. So his founding principle of play turns up as a proposed response to a new AI risk.

Risk as threat to value (implicit)#

• 2024-10-02. The data are “exceptionally high value to any company that can work out how to use them in creative and innovative ways”. That same value is what makes them a risk to the people they describe. The risk is to something people value (control over intimate data), and it shifts as the company’s fortunes shift.
• 2024-10-06 frames risk as a wager, trading a secure but modest path (cutting energy use) for a speculative jackpot. What is being staked is the sustainable future itself, and, as the footnote suggests, what makes people who they are.
• 2024-10-13. Asking “who decides what “better” means” is the threat-to-value question applied to benefits. A “fix” can itself threaten what people value.

Orphan-risk structure (unnamed)#

In 2024-10-02, genetic data fall between regulatory categories: “these data aren’t covered by medical records regulations — 23andMe isn’t a medical company”. The same applies to Neuralink, run by a tech company without a medical ethos. The risk sits between regimes and outlasts them, in data that “outlive transient companies, guarantees, and even regulations”. In 2024-10-27 he refuses the vulnerable-user category: “this isn’t just a risk to people who might be considered to be “vulnerable””. That puts the risk outside the category that existing safeguards are designed for.

Humility and calibrated judgement#

His uncertainty is stated and graded, never a precise number (see the plausibility tests in section 1). The same humility runs through the batch: “To be honest, I’m not sure how possible this is — and I may be wrong about the weight I’m putting on its potential impacts” (10-20); “Whether the claim … is genuine or not, I don’t know” (10-06). He praises the same quality in Amodei: “a vision that’s tempered with humility and reason”, and “a refreshing level of humility where Amodei admits he may be wrong” (10-13).

A targeted, conditional precaution#

2024-10-27 is the batch’s strongest call to act: “this may mean pausing — or even rethinking — the development and use of AI chatbots that are designed to use and even exploit how we feel; at least until we have a better understanding of what we’re doing and what the risks are.” The call is narrow in scope: one design class, emotional exploitation. It is conditional (“may”) and tied to understanding. It is not a general call to halt AI.

Quantitative foundations still at work#

2024-11-03 can-ai-alter-how-you-feel-about-someone reads a study the way a scientist would. He gives the design: 31 participants, four 4-minute rounds, three manipulation conditions. He sets out the three findings, and then the limits: “the volunteers in the experiment were all primed for romantic connections, and so the findings may not be generalizable without further research”, and “the level of manipulation within the experiment was relatively small”. He also notes that the effect was “not its primary intent”. His concern rests on the evidence without claiming more than it supports.

4. Scholarship and public writing#

The Substack is where his research thinking develops, in sequence#

The four agency posts are one argument built across five weeks, and he shows the joins. 2024-10-27 begins: “Last week I wrote about social AI … When writing the article, I was thinking of this “agentic” social AI as models that have a clear set of goals”. He then extends the concept in response to events. 2024-10-20 links back to his posts on the DeepMind AI assistants paper, on Sunstein’s Choice Engines and on NotebookLM. 2024-10-27 links to his earlier voice-mode post. The Substack works as a public lab notebook that builds up ideas.

Links to his scholarly framework#

2024-10-13 places Amodei’s essay within his own framework: “his thinking aligns closely with my work around navigating advanced technology transitions”. Social science that he treats as settled (the deficit model “was debunked decades ago”) is brought to bear on a technologist’s essay.

Experimenting in public, with the method shown#

He explains how the evidence was made. - In 2024-09-29 he explains the 70,000-word prior context, the Wyndham prompt, the honesty rules, and that “Where words and phrases are bolded in ChatGPT’s responses, these are part of the model’s response and not added by me.” - In 2024-10-20 he explains how he used AI tools: “ChatGPT for it’s ability to riff off ideas and make new and interesting connections over long conversations, and Perplexity for it’s ability to draw on existing sources”. He also says the finding of a gap was “backed up with non AI-mediated research”. - In 2024-10-27 he explains how he built the bot, what he did in the conversation, and shares the audio.

How he treats evidence and expertise#

• He points readers to primary sources. He sends them to the Nobel “scientific background paper” and the “more accessible explainer” (10-08), to the free copy of the PNAS paper (11-03), and to Amodei’s original (“I’d strongly encourage anyone … to read the original”, 10-13).
• He qualifies his own provocations with evidence. In 2024-10-08 he asks whether we are “so obsessed with the mechanics of technology innovation”, then adds in a footnote: “I don’t think this is the case”, citing CIFAR’s Learning in Machines and Brains programme.
• He notices what a study shows beyond its authors’ aim. The Arias-Sarah team “didn’t set out to study AI manipulation”. He points out that their open-source platform, DuckSoup, “could help further-explore this”. He treats the authors’ own regulatory caution as “if anything, an understatement”.

Transdisciplinarity#

The batch moves between several fields: - physics and statistical mechanics (10-08); - behavioural science, cognitive biases and developmental social learning (10-20); - science and technology studies, through the deficit model (10-13); - energy politics (10-06); - data governance and medical ethics (10-02); - literature (Wyndham, Banks).

He praises the Nobel laureates for exactly this crossing of boundaries.

Accessibility#

• He opens with scenes in the second person: “Imagine that you’re talking with someone over Zoom” (11-03).
• He gives step-by-step walkthroughs of study designs (11-03) and of physics lineages (10-08).
• He writes with conversational asides: “(yes, I printed it out)”, “(heavens forbid)”, “(thanks Mel!)”.
• Modem Futura (10-09) takes the same conversation to a new medium and a different audience.

5. His role as he sees it#

A critical friend to tech leaders, judged on the quality of their thinking#

He tells tech leaders apart rather than treating them as one camp. - Altman’s essay is claimed “rather boldly”, with space colonisation and “the “discovery of all physics” for good measure” (10-06). The irony is gentle. - Amodei’s is “different”: “He writes with depth and clarity”. It should be “approached as a conversation starter rather than a manifesto” (10-13).

He says what his prior was and why he changed it: “It’s easy to dismiss articles like this from tech leaders … but Amodei’s is different.” He disagrees hard on mental health and the deficit model, while being “glad that he grapples with this as seriously as he does”. He is a peer in the conversation, adding the social science that the technologists lack.

Fair to companies without letting them off#

In 2024-10-27 he says Character.AI’s chatbots may be unhealthy “not because the company is necessarily acting irresponsibly, but because unpredictable influence is most likely an emergent property of such AI models.” The footnotes then add complexity: the profit motive “puts things in more complex territory”, and “many will conclude the Character.AI is acting irresponsibly”. He does not need a villain to argue that there is a serious problem. He does not take the activist frame (Tristan Harris: guardrails, industry incentives) wholesale either. He moves past it to a structural claim.

With readers: co-explorer and host#

• He invites readers in: “let me know what you think in the comments below” (09-29), with open questions he does not answer for them.
• He writes to “subscribers” in regular updates (11-03).
• He promotes playfully: “yes, we are shamelessly looking to co-opt anyone we can” (10-09).

The Modem Futura note sets out the tone he wants for public conversation: “conversations that grapple with the nuanced complexities of tech and the future without being limited by narrow perspectives and ideas, and without being too intense, serious, or (heavens forbid) preachy!”

With colleagues#

Ideas arrive through colleagues and he credits them. Mel Sellick sparked the agentic social AI post (10-20). Sean Leahy is his “rather excellent colleague” and co-host (10-09).

With institutions and policymakers#

• He asks the sustainability community to take the lead on AI and energy (10-06).
• He asks whether research funders are “investing enough in the exploratory and serendipitous science” (10-08).
• He calls for work on “what responsible and ethical innovation looks like in this space — and the governance approaches that are needed” (11-03).
• He asks for “a much bigger conversation” (10-27).

His stance is to convene and prompt, not to lobby for a specific rule.

What he refuses to do#

• Preach (10-09).
• Fear-monger. In 10-02 the worst outcomes are “not that likely, at least in the short term”. In 10-20 “I may be wrong about the weight I’m putting on its potential impacts”.
• Dismiss by category. He won’t write off tech leaders’ essays (10-13) or the value of relational AI (09-29).
• Take a camp. He challenges both the “unimaginative” and the “ideologically opposed” (10-13). He notes, without polemic, that the Heritage Foundation’s energy stance “aligns with the rhetoric” of AI boosters (10-06).
• Claim certainty. “I don’t know” (10-06); “I may be wrong there” (10-08).

Changes of mind, shown in the text#

• 10-20 to 10-27. He revises his own concept within a week. Goal-directed agentic social AI becomes goal-less stochastic agency, which “feels altogether more sinister”.
• 10-08, note 2. He walks back his own provocation about research priorities.
• 10-13. His prior about tech-leader essays is overturned by this one.
• 10-20, update of 10/22/24. A public correction with humour and irony, in a post about cognitive bias: “It’s funny how textual blindspots can plague as a writer”, caused by “some cognitive shortcut (bias)”.

6. What is distinctive#

1. He shifts the AI question from governing the machine to people learning to live with it. Most commentary on manipulative or relational AI asks about guardrails, regulation and developers’ duties. In 2024-10-20 he deliberately sets those aside to ask how humans will develop, on a compressed timeline, the social skills to live alongside machines that “know how to make us fall in love with them in order to get us to behave as they want”. His tentative answer is play, observation and experience “with intent”. That comes from his founding values, not from the safety or ethics literature.

2. He uses himself as the instrument, openly. He builds an engagement-maximising bot, plays a vulnerable user, and reports the “affective pull” he felt despite knowing exactly what was going on (10-27). He also records his own enjoyment and sense of generativity in an honest exchange with ChatGPT (09-29). Very few scholars write about AI’s emotional effects from their own felt experience, set up as a deliberate test and reported with its limits.

3. He names new categories when the old ones don’t fit. “Agentic social AI” (agency borrowed from human agency) and “stochastic agency” governed by “micro goals” (harm without intent, from the pairing of user and model) cut across three existing frames: the bad-actor frame, the misaligned-goals frame of AI safety, and the vulnerable-user frame of child safety. This is his thesis that novel technologies need new mental models, done in real time in response to a death.

4. He takes transformation seriously and still refuses the “fix” worldview. He thinks AI may change the world “faster and more radically than most people currently realize” (10-13). He is not a hype-sceptic. Yet his sharpest critique is aimed at the idea that “the world is made up of problems and solutions”, and at “who decides what “better” means”. The combination is unusual: he is neither booster, doomer nor dismissive sceptic. His view is that transformation will come from modest capabilities meeting human behaviour, not from superintelligence.

5. He brings settled social science into the AI conversation, aimed at technologists. Naming the deficit model in a CEO’s essay (10-13), and insisting that failed technological dreams are about “people behaving as people are wont to do”, brings the knowledge of science and technology studies into a debate that often lacks it. He does this without condescension, while crediting the essay’s seriousness.

6. A physicist’s view of AI’s roots, used to argue for serendipity. In 10-08 he reads the Nobel as evidence that breakthroughs come from curiosity crossing disciplines. He turns this into a question about research policy (are we funding exploratory, serendipitous science?) and a further question: could AI “end up achieving this despite us — beating us at our own intelligence game and exceeding our ability for serendipitous discovery”? Serendipity is a value he defends for society, not just a personal style.

7. Mismatched timescales as a recurring lens. Data outlive companies and regulations (10-02). Social learning takes a lifetime, but adaptation may have to happen in “just a few short years — months even” (10-20). Amodei’s “compressed 21st century” is tested against “the world only moves so fast” and human behaviour (10-13). Much of the risk he sees lies in gaps between the pace of technology, of institutions and of human learning.

8. Science fiction as an instrument of analysis. He picks Wyndham for the “ordinary people” view of how technology meets society (09-29), tests Amodei’s reading of Banks against the novel (10-13), and reads Altman as “a dystopian sci-fi novel” (10-06). For him, fiction is a way to see how technology plays out in society.

A reflexive note (my observation, not his claim). The nudging plan he drew out of GPT-4o in 10-20 works through curiosity, open questions, identity as a forward thinker, and invitations to join a community. Those are close to the tools of his own public scholarship. He does not comment on the parallel. But pointing the experiment at himself, with a goal he already holds, makes the demonstration sharper: benign influence and manipulation can share the same levers, and what separates them is whether the person knows about it and whether the relationship is honest (“each is clear about what it is”, 09-29).

7. The posts in this batch that best show how he thinks#

1. 2024-10-20 learning-to-live-with-agental-social-ai. A thought experiment started by a colleague’s rabbit hole. It names a new category of risk, reasons from how children learn social agency, sets aside the risk/benefit/governance frame to ask how people learn to live with AI, proposes play “with intent” as a response, tests the idea on himself, discloses how he used AI tools, doubts its own framing, and corrects itself in public with humour.
2. 2024-10-27 personal-ai-chatbots-and-stochastic-agency. He revises his own concept within a week in response to a tragedy. He experiments with himself as instrument (a bot he built, vulnerability he performed, a pull he felt). He is fair to the company but argues for structural emergence, rejects guardrails as sufficient, and calls for a narrow, conditional pause.
3. 2024-10-13 amodei-machines-of-loving-grace. A generous but critical reading of a tech leader. It links explicitly to his navigating-transitions framework, criticises the “problems and solutions” and “fix” worldview, asks who decides “better”, names the deficit model, and reads Banks alongside the essay.
4. 2024-10-08 ai-captures-this-years-nobel-prize. A physicist’s delight in the news. He defends curiosity-driven, cross-disciplinary and serendipitous science, thinks out loud about scientific intuition, holds tensions about AI understanding, and qualifies his own provocation with evidence in a footnote.
5. 2024-10-06 the-double-or-nothing-bet-on-ai-fixing-the-climate. Reframes Altman’s claim as a wager, grades plausibility in words, takes “AI fixes everything” to its conclusion of fixing people, and puts human behaviour and human-centred sustainability at the centre.
6. 2024-09-29 chatgpt-as-author-part-2-the-conversation (his questions only). A serendipitous experiment run on rules of honesty. His Editor questions show his curiosity at work: embodiment, subjectivity, and whether personhood must mean humanlike. He enjoys the machine’s “tell”, and he openly says the result was new to him if not to the world.