B10 digest: 2023-07-25 to 2023-09-01 (18 posts)#

What this batch is#

Late summer 2023. Maynard has just taught his first ChatGPT course, term is about to start, and he is serialising audio readings of Films from the Future as The Moviegoer’s Guide to the Future (episodes 4–9; only his short intros are in the text). There are no Modem Futura podcast posts.

Not his: a guest post by Brad Allenby and a cross-post stub of Mark Daley’s essay. Mixed provenance: ChatGPT-generated terms (provenance-of-ideas), a class-written letter (Musk), questions compiled with colleagues (fifteen questions).

The centre of gravity is AI in learning and cognition. Around it run a compact AI consciousness and personhood thread and an AI–nuclear comparison.

Main ideas#

1. Generative AI as a catalyst for thinking, with flaws as a feature (2023-08-14). From reading over 2,000 student–ChatGPT conversations: ChatGPT is “a profoundly effective catalyst for engaged and creative thinking”, sometimes “because of its limitations”. Its unreliability forces users to question and test, “at least if they understand what they are doing”. The educational point is learning how to think, not what to think, so accuracy-obsessed critiques miss “the point”.

He hedges with self-irony, reports critics (Marcus; the Rolling Stone feature on Gebru, Buolamwini and others) respectfully, and names “moral as well as social and economic” risks. Here AI’s effect on cognition is framed almost entirely as amplifying thinking; the danger is “not smart” use and missing AI literacy, not cognitive erosion. A clear benchmark for later batches.

2. Equity, access and institutional responsibility in education. Three posts turn this into practice: - Admissions (07-27). AI can level admissions by acting as a “translator” of students’ jumbled thoughts into prose. That breaks the grip of coached, class-coded essays. It needs universal access (the digital divide) and AI literacy in “every high school”. - Fifteen questions (08-02). He poses questions for professors but refuses to answer them. Answers are “the responsibility of instructors and their schools and colleges”. Opting out (Q4) and academic freedom (Q5) sit alongside AI literacy (Q15). - ChatGPT Enterprise (08-29). Enterprise could solve universities’ privacy and IP barriers and the $20 equity gap. It would become “a forcing function in how we teach”, good for students with “jagged profiles”.

He is strongly pro-adoption and treats an OpenAI product as the enabling solution; corporate dependence is not raised. Risk is framed as user naivety and uneven access.

3. AI consciousness, personhood and the ethics of control (08-18, 08-23, 08-25). A compact but important cluster: - The Ghost in the Shell intro warns against “dehumanizing” future AIs by denying they are or could be “human” in order “to justify how we control and use them”. - The consciousness post goes further than the Butlin/Long paper it reviews: - he endorses computational functionalism (“stands up to scrutiny”) and thus substrate independence; - he names the tension between “the economic expediency of denying consciousness” and the duty not to inflict suffering; - he broadens suffering to include loss of freedom, agency and sense of self; - he calls for societal norms, policy and regulation, “more than just research”; - he rules out ways of “enslaving AIs” justified as “just machines”. - The Ex Machina intro reverses the direction: AIs with “awareness and agency”, outside human morality, might leave humans “manipulated, used, and enslaved”.

He holds both directions of domination in view, and separates consciousness from existential-risk “flights of fantasy”.

4. How ideas form, in heads and in models (08-21). A personal discovery drives the post: his “future of being human” framing predates the 2021 CIFAR call and goes back to a 2017–18 chapter he had forgotten. He argues that human idea formation is non-linear, collective and “something of a black box — even to me”. He proposes an “ideas mycelium” metaphor and applies it to LLMs, whose outputs fruit from a “mat of associations” with little traceability.

Governance attention shifts from input–output attribution to the “shape, structure, and composition” of inputs, which drive bias and need oversight where necessary. Consent and copyright over training data are acknowledged as “important questions”. He also states that generative AI is now “part of the mix of influences that inform and modulate our thinking”. An early, unalarmed statement of the AI-and-formation theme.

5. AI compared with nuclear weapons (07-25). The comparison is structural and literal. Technology trajectories are “deeply intertwined with very human power dynamics, politics, and personal beliefs”. But AI is “hidden, dispersed, readily accessible”, and its risks are driven by “shadowy actors and naive developers”, unlike “contained” nuclear programmes. Nuclear weapons are “a more tangible and immediate risk”. Named risks: democracy and truth, a US–China arms race, weaponisation. He wants nuanced conversation, not “bumper sticker” stances.

6. Method and responsible innovation. The recurring Films from the Future boilerplate rejects polarised doom/salvation narratives and prefers “dialogue and discussion” to “preaching”. The Musk anecdote (08-28) frames responsible innovation as the gap between what entrepreneurs “can” and “should” do. The 08-08 repost of his 2021 National Academies talk re-endorses the risk innovation framework as “perhaps more relevant than ever in an age of generative AI”. He hosts Allenby’s complex-systems critique of “a Gosplan for AI” (not his words).

Concepts appearing#

AI as catalyst for thinking; flaws as a learning feature; universal AI literacy; AI as “translator”; equity of access and the digital divide; institutional responsibility; pedagogical forcing function; the ideas mycelium; training-data composition and bias; computational functionalism; over- and under-attribution of consciousness; economic expediency versus moral responsibility; dehumanisation and enslavement of AI; personhood beyond the human; a hidden, dispersed risk landscape (versus nuclear); power dynamics in technology trajectories; can versus should; risk innovation; anti-polarisation method.

What is new or changed#

• A rapid shift on machine consciousness. Within a week he moves from “a long way from machines that have consciousness” (08-18) to finding near-term arguments “compelling” (08-23) to “ever-closer” (08-25). Accepting substrate independence also sits uneasily with the 2018 hardware-based scepticism about human-like machine minds, republished in April 2023. He does not comment on that tension.
• Peak educational optimism. Classroom experience turns the April 2023 enthusiasm (AI literacy, universities as adopters) into strong claims about AI improving thinking and equity. Worry about corporate dependence or cognitive offloading is largely absent.
• Human–machine symmetry in cognition. He is newly explicit that LLM “assimilation” resembles human idea formation. This softens strict attribution while keeping consent and bias concerns.
• Continuity. He re-endorses risk innovation (2021 talk) and the 2018 manipulation and permissionless-innovation chapter (“probably more so now”). He stays measured on existential risk (nuclear is “more tangible and immediate”; he separates consciousness from x-risk “fantasy”).

Most important posts#

1. 2023-08-14 chatgpt-stimulates-creativity-critical-thinking. His fullest statement on AI and learning, flaws-as-feature and AI literacy.
2. 2023-08-23 could-we-build-conscious-ais-in-the-future. AI moral status, computational functionalism, the ethics of control, governance “more than just research”.
3. 2023-08-21 the-messiness-of-the-provenance-of-ideas. Human and LLM idea formation, the ideas mycelium, AI as an influence on thinking.
4. 2023-07-25 oppenheimer-and-ai. The AI–nuclear comparison: power dynamics, a diffuse risk landscape, anti-bumper-sticker discourse.
5. 2023-08-18 being-human-in-an-augmented-future (read with 2023-08-25 ai-platos-cave). Dehumanising AI, personhood, and his 2023 updating of the manipulation thesis.
6. 2023-07-27 chatgpt-and-college-applications. AI as translator and equaliser, the digital divide, universal AI literacy.