Late Lessons, Jensen Huang and AI
Markdown version

B03 reading notes: 14 posts, 2018-11-25 to 2019-06-12#

Batch context. This is the Medium-era archive, carried over to Substack, from the months after Films from the Future: The Technology and Morality of Sci-Fi Movies came out (Mango, November 2018). Several posts are republications or reworkings of pieces he first published in The Conversation (one dates from December 2015) or OneZero, or are extracts from the book. The book is sole-authored, so its extracts are his own prose. No post in this batch contains AI-generated text, guest text or co-written text. No Modem Futura podcast posts fall in this batch. One post (2019-02-08) has a header image credited to Midjourney, which must have been added at a later edit because Midjourney did not exist in 2019. It has no bearing on the prose.

Relevance: high 5 posts, medium 7, low 2.

Title: Twenty Trends in Science and Technology from Twelve Science Fiction Movies Relevance: medium

Provenance: His own prose. Each of the 20 trends gets a short framing paragraph and a block excerpt from Films from the Future, which he wrote alone. The post is promotional.

Argument, in his terms: This is a catalogue rather than an argument. The book, he says, has “a lot to say about the challenges of getting advanced technologies right”, and it uses 12 films to open up 20 areas. They are de-extinction, complex systems, cloning, crime prediction, ubiquitous surveillance, smart drugs, the nature of intelligence, bioprinting, automation, human augmentation, AI, superintelligence, synthetic biology, technological convergence, nanotechnology, gain-of-function research, resilience, geoengineering, Occam’s razor and extraterrestrial life. The parts that bear on the map: - AI. Deep learning and natural language processing have “revitalized” AI, and “the future with AI will be very different from the present.” The book’s Ex Machina chapter sets his priority. We should worry less about checks and balances against superintelligence and more about AIs that exploit human cognitive vulnerabilities. We also need tests that show when we are being manipulated by machines. This is the earliest and clearest statement in this batch of the cognitive-vulnerability and manipulation framing. - Superintelligence. He calls himself “something of an agnostic”. He recalls believers at scientific meetings trying to convert him, and likens the experience to a religious convention. He doubts the idea is plausible and distrusts the near-religious way it is promoted. - The nature of intelligence. Our idea of intelligence is shaped by human self-importance. That shapes how we think about technologies meant to enhance it (smart drugs) or build it (AI). - Crime prediction and surveillance. Machine learning in policing raises “tough questions around the decisions we leave to machines, and what it means to be labeled as ‘good’ or ‘bad.’” Data from the Internet of Things (IoT) can do good, but it lets corporations and governments “control their lives in new ways”. - Automation. Advances in robotics and AI are “promising (or threatening, depending how you look at it) a step-change”. - Convergence. “The most profound shifts in capabilities” happen where biology, digital technology and materials meet. - Resilience. He gives it a broad meaning: it “gets to the heart of how we think about what’s important to us” and how we protect and grow that. This lines up with his risk-as-threat-to-value framing. - Occam’s razor. He offers it as a tool for sifting “grandiose claims and fearful projections”, which is his anti-hype heuristic. - Education. In an informal poll of one of his undergraduate classes, over 50% admitted using smart drugs such as Ritalin, Adderall and Modafinil.

Analogies and comparisons: Films serve as a lens throughout. Biotech, nanotech and AI appear side by side as converging domains rather than as analogues of one another. Nanotech is framed as a continuation of materials science, which is his own home field.

Quotes (his prose, book extracts): - “guarding against AIs that learn how to use our cognitive vulnerabilities against us” - “develop tests that indicate when we are being played by machines” - “I sometimes had to remind myself that I was at a scientific meeting, not a religious convention” - “how we think about intelligence is remarkably colored by our sense of our own importance”

2018-12-13 · tech-startups-orphan-risks#

Title: It’s time for tech startups and their funders to take “orphan risks” seriously Relevance: high

Provenance: His own prose. Later posts link to LinkedIn and Medium (the edge-of-innovation publication) versions of the same piece.

Argument, in his terms: - The gap in Rumsfeld’s taxonomy. Rumsfeld split uncertainty into known knowns, known unknowns and unknown unknowns. That scheme misses risks that are “known knowns” if you’re looking in the right place but are not taken as seriously as they should be. - Orphan risks defined: “risks that are perceived as being too ill-defined, too complex, or too irrelevant to be worth paying attention to, yet have the power to derail entire enterprises down the line”. He says “the concept of orphan risks is a new one, although the underlying reality is not.” In 2018 he presents the term as newly introduced through the ASU Risk Innovation Lab. - What falls into the category. Ethically ambivalent behaviour, threats to deeply held values, and emerging technologies “perceived to come with highly unusual and uncertain dangers”. These are mainly social risks that resist quantification and standard risk management, and their messiness is why they get overlooked. Examples include discrimination and exclusion, threats to social equity and livelihoods, and lost public trust. Uber and Facebook are his cautionary cases. - Why it matters now. A “growing chasm” separates “tried and tested ways of thinking about risk” from the risks now tripping companies up. AI, gene editing and advanced manufacturing, together with an interconnected global society, are reshaping the “risk landscape that lies between new ideas and their successful implementation.” - Risk innovation, “simply put”: to thrive amid innovation, “we need parallel innovation in how we think about and act on risk.” This is the founding idea of the Risk Innovation Lab, which he directs. - Risk as threat to value: “a threat to something of importance to an individual, a community, or a business organization.” He admits it is “somewhat subjective”. Its strength is that it brings in things that are hard to quantify: “autonomy, dignity, trustworthiness, self-esteem, way of life, and even deeply held beliefs and convictions.” Health, wealth and the environment “fit comfortably into the ‘value’ bucket”. It “extends conventional thinking rather than replacing it” and captures “the reciprocal dangers of threatening what is important to others through what they do.” - Governance. “Few of these orphan risks are overtly governed by laws and regulations.” His remedy is not regulation but up-front attention and tools for startups and investors, through the ASU Risk Innovation Accelerator. - How he makes the case. Mainly on business success (being “blindsided”), with an ethical point alongside (“quite apart from the ethical inappropriateness of ignoring actions that could present deeply impactful social risks to others”). He is confident and writes as an adviser to business.

Engages: Donald Rumsfeld; Kara Swisher (“Silicon Valley needs more adult supervision”); Mitchell Baker of Mozilla, on technologists’ blind spots.

For the map: This is the 2018 origin of “orphan risks” in his public writing. At this point it is a startup and investor risk tool nested inside risk innovation and threat-to-value. It is not an AI-specific concept.

Quotes: - “risks that are perceived as being too ill-defined, too complex, or too irrelevant to be worth paying attention to, yet have the power to derail entire enterprises down the line” - “we need parallel innovation in how we think about and act on risk” - “a threat to something of importance to an individual, a community, or a business organization” - “This way of thinking about risk extends conventional thinking rather than replacing it”

2018-12-15 · if-elon-musk-is-a-luddite-count-me-in-6edc2e786756#

Title: If Elon Musk is a Luddite, count me in! Relevance: high

Provenance: His own prose. It was republished from The Conversation (23 December 2015), so it records his 2015 thinking. “December 21” in the opening refers to 2015.

Argument, in his terms: - The ITIF awards. The Information Technology and Innovation Foundation (ITIF) nominated Musk, together with Hawking and Gates, for a 2015 “Luddite Award” because he had voiced concern about the dangers of AI. ITIF is right that innovation drives growth, but “what it misses by a mile is the importance of innovating responsibly.” His section heading is “Being cautious ≠ smashing the technology”. - Historical grounding. He cites the European Environment Agency (EEA) report Late Lessons from Early Warnings, which he dates to 2002, and its 2013 follow-on. In his words they catalogue innovations “from PCBs to the use of asbestos” where early warnings were “ignored or overlooked”. He links this to climate change, pollution and industrial chemicals as consequences of “unfettered innovation”. (I record this only as his own citation and make no comparison.) - What is new about emerging technology. AI, robotics and the IoT make future risks and benefits harder to see, especially where the technologies converge (the “Fourth Industrial Revolution”). He quotes Klaus Schwab on inequality and the displacement of labour, and notes that even this optimist sees social complexity. “For the first time in human history”: we engineer matter atom by atom, reprogram DNA, “aspire to creating artificial systems that are a match for human intelligence”, and connect people and devices faster and in more complex ways than before. These challenges “cannot be brushed off by assuming business as normal.” - Complexity. Like any complex system, the explosion of capability is “likely to look great… right up to the moment it fails.” - Responsible innovation as a movement. He cites ASU’s School for the Future of Innovation in Society (SFIS), the EU’s Responsible Research and Innovation (RRI) programme, and the 2015 human gene-editing summit. The summit made clinical germline use conditional on safety and “broad societal consensus”, and he reads that as “responsible scientists”, not neo-Luddites. - Criticism of Musk. Musk “sometimes” does not go far enough. His answer to his AI fears, OpenAI, accelerates AI development “in the hopes that the more people are involved, the more responsible it’ll be”. That still assumes “the answer to technology innovation is… more technology innovation.” This is an early criticism of a founding AI-lab strategy, made in 2015. - Justice. Past inventions look good “especially if you’re privileged and well-off”. Innovation must improve “the lives and livelihoods of all — not just the privileged.”

Views on AI: He treats the AI worries of Musk, Hawking and Gates as legitimate reasons to ask what could go wrong (he links a Conversation piece whose URL reads elon-musk-is-right-we-need-to-talk-about-artificial-intelligence). Human-level AI is listed as one of several unprecedented capabilities.

Quotes: - “what it misses by a mile is the importance of innovating responsibly” - “it still adheres to the belief that the answer to technology innovation is… more technology innovation” - “it’s likely to look great… right up to the moment it fails” - “not simply innovate in the hope that things will work out OK in the end”

2019-01-16 · responsible-innovation-entrepreneurs#

Title: Tech businesses need to think differently about ethical and responsible innovation Relevance: medium

Provenance: His own prose. It is based on his Conversation article “Sci-fi movies are the secret weapon that could help Silicon Valley grow up”, and it overlaps heavily with the 2019-03-07 post. It includes book promotion. The final sentence is cut off in the original.

Argument, in his terms: - Good intentions are not enough. Companies need “a sophisticated understanding of the often complex dynamic between technology and society.” Mitchell Baker is “right in part” that technologists should engage the humanities, but that alone is not enough. Firms need to draw on expertise in “socially responsible innovation” (the SFIS community) and “escape the ruts of conventional thinking”. - Dual use. A Google/DeepMind AI system reads lips better than people can. It could help people who have trouble speaking aloud, but combined with long-range cameras it could threaten the privacy and security of millions. Other cases he cites are Facebook, Uber and Musk’s trouble with the Securities and Exchange Commission (SEC). - Concepts. He uses “social risk landscape” and “orphan risks” (“what we are increasingly calling”). He also uses “permissionless” innovation in scare quotes, which he glosses via Jurassic Park as innovation “driven by power, wealth and a lack of accountability”. So permissionless innovation is a pejorative for him here. - AI through Ex Machina. The film shows “a wealthy and unaccountable entrepreneur who is supremely confident in his own abilities”. It also shows the danger of “machines that know us better than we know ourselves, while not being bound by human norms or values.” Minority Report stands in for AI-enabled crime prediction. - Tech leaders. The recurring figure is the wealthy, unaccountable, overconfident founder, such as Hammond and Nathan in the films, and he answers it with humility. - Publics. Sci-fi films are “social and educational levelers”. They bridge people who know how technology works and people who know what makes it serve society.

Quotes: - “machines that know us better than we know ourselves, while not being bound by human norms or values” - “‘permissionless’ innovation that’s driven by power, wealth and a lack of accountability” - “without humility and a good dose of humanity, our innovations can come back to bite us”

2019-03-01 · personal-fitness-tracking-and-the-hidden-risks-of-seductive-technologies-4070bf17897f#

Title: The Forgotten Risk of Fitness Trackers Relevance: medium

Provenance: His own prose.

Argument, in his terms: - The case. The Fitbit Inspire is sold to employers and insurers. Wearers may be “handing over control of your life to someone else”. The data could decide perks, jobs and medical bills, could be used in prosecutions, and could be hacked (Fitbit was hacked in 2016). Adoption keeps growing “in part because consumers are easily seduced by the promise of new tech.” - Orphan risks in consumer technology. He applies the concept here and defines it as “risks that don’t seem important or are easily overlooked but can come back to bite”. Examples include not tracking “the changing landscape around tech governance” and users “feeling threatened by how their data might be used against them”. - Reciprocity. Consumer trust can turn into its own orphan risk of backlash if users feel exploited: “These kinds of risks aren’t a one-way street.” - Remedies. Manufacturers should engage users early, before it is too late to change course. Consumers must ask critical questions and not leave them to others, which would leave them “wide open to being abused”. Trustworthy sources of information are scarce. Creative media such as sci-fi films help “as long as they’re accompanied by a trustworthy guide”. He also points to responsible innovation and public-interest technology. - How he makes the case. A “win-win” for company and user. The risk is harm to users and risk to the business at the same time.

Quotes: - “consumers are easily seduced by the promise of new tech” - “risks that don’t seem important or are easily overlooked but can come back to bite” - “These kinds of risks aren’t a one-way street.”

2019-03-05 · should-we-be-treating-algorithms-the-same-way-we-treat-hazardous-chemicals-e39b5d02112c#

Title: Should we be treating algorithms the same way we treat hazardous chemicals? Relevance: high (the centrepiece of the batch)

Provenance: His own prose. At about 4,300 words it is the longest analytical piece in the batch.

Argument, in his terms: - Framing. Algorithms and hazardous chemicals are physically unlike, yet “both have an uncanny ability to impact our lives in ways that are neither desirable, nor always easy to predict.” The field of “algorithmic risk” is immature, and chemical risk assessment could “accelerate progress”. - The spectrum of AI risk. At one end sits the “well-publicized yet rather speculative risk of superintelligent machines wiping out life on earth”. At the other are closer-to-home risks such as self-driving cars deciding who lives in a crash and bias inherited from creators. - The trigger case. A paper on “predictive inequity” (arXiv 1902.11097) raised the possibility that pedestrian-detection systems detect darker-skinned people less well. He contrasts the MIT Technology Review headline and Kate Crawford’s tweet with the more guarded paper. His lesson: “knee-jerk reactions to seemingly-startling results rarely result in socially beneficial outcomes.” - The chemical side. Commercial chemicals are “the backbone of modern society”, yet if misused they can “disrupt the climate, destroy ecosystems… cause disease and death”. Businesses and regulators “had to learn the hard way”. He admits chemical risk assessment has limits: it still struggles with chronic exposure, endocrine disruptors and exposure during critical developmental windows, and its methods “are not nearly as robust as we’d like them to be.” - Five concepts to transfer: 1. Hazard versus risk. Conflating them “doesn’t help anyone”. His illustrations are bleach, gasoline and a grizzly bear seen at a distance versus met face to face. A flawed algorithm that could control aircraft is a hazard; it becomes a risk only if deployed without safeguards. 2. “Algorithmic exposure.” “Users of systems and devices are effectively exposed to algorithms if they are affected by the decisions those algorithms lead to.” This extends to anyone affected, not only users. Exposure can be uneven, weighted toward darker-skinned pedestrians. In the predictive inequity case the lab algorithms differed from on-road systems that use LIDAR, so there is “no clear algorithmic exposure route”. That does not negate the research, but it shows why “viable exposure pathways” matter. 3. Consequences. The type of harm matters as much as its probability. For algorithms, harm goes beyond health: “how do we begin to parse the consequences of algorithms that result in people’s livelihood being threatened? Or their liberty, dignity, and self-respect?” 4. The “algorithmic exposure-response relationship”, which is his coinage and parallels dose-response. It gives a number for decision-makers but does not settle what is acceptable. 5. Weight of evidence, checks and balances, and freedom from bias. Examples are the Cochrane Handbook, scepticism about single studies, attention to vested interests, and avoiding cherry-picking. His examples of cherry-picking are anti-vaccination communities and “fear around the use of genetic modification in food”. - Acceptable risk. “The ideal is that there’s no such thing as acceptable risk”, but with chemicals, society has accepted there is “no such thing as zero risk”. Also “there’s always a risk of not developing or using a product”. On skin-tone bias “there’s a very strong case” that no risk is acceptable. Yet human drivers show a similar bias, so there is still a trade-off with the status quo. - Conclusion (firm). Similar rigour “must” be applied, “lest we end up building our understanding of algorithmic risks on an evidentiary stack of cards”. He calls for checks and balances, evidence-based decisions, and attention to easily overlooked risks such as racial, gender and religious discrimination.

Nature of the analogy: Structural and methodological, not literal: “Of course, an algorithm is not a chemical… Yet like many chemicals, algorithms have the potential to endanger lives.” Once “different mechanisms” are set aside, he says, the analogy “becomes intriguingly compelling”. He means the transfer of method (hazard, exposure, consequence, dose-response, weight of evidence) literally, as practice. GMOs and vaccines serve as examples of evidence misused.

Engages: Kate Crawford and the AI Now Institute (he praises AI Now’s “sterling work” and also criticises the over-reading on social media). Also Deloitte on algorithmic risk management, the Cochrane network and MIT Technology Review.

Views on AI: AI here is “algorithms”, a source of risk to be assessed. The risks he stresses are bias and discrimination, safety in autonomous vehicles, and harms to livelihood, liberty and dignity. Superintelligence is treated as speculative. He writes as a risk scientist committed to evidence and wary of both complacency and alarm.

Quotes: - “knee-jerk reactions to seemingly-startling results rarely result in socially beneficial outcomes” - “Users of systems and devices are effectively exposed to algorithms if they are affected by the decisions those algorithms lead to.” - “lest we end up building our understanding of algorithmic risks on an evidentiary stack of cards that collapses at the first hurdle” - “Unless, that is, we’re content to repeat the mistakes of the past as we reinvent the future.”

2019-03-06 · navigating-the-risks-and-benefits-of-new-technologies-5884bc324d63#

Title: Navigating the risks and benefits of new technologies Relevance: high

Provenance: His own prose. This is the closing chapter of Films from the Future: it refers to “the previous chapters” and to “wrapping up a book”, and the image is credited “©2018 by Andrew Maynard”. He wrote it on the Isle of Arran, where he first went in 1984.

Argument, in his terms: - Nostalgia versus obligation. Arran’s slow pace comforts him. Happiness lies in “food, shelter, warmth, and good company”. But nostalgia is a “sentimental illusion”. Technologies “developed and used responsibly, can and do improve lives”. Renouncing them from privilege denies others the choice, and “we have an obligation to explore new ways of using science and technology to improve the world”. That obligation carries “tremendous responsibilities”, both to benefit people without harming them and to “live responsibly” in a world we keep changing. - Who decides. It is “all too easy to leave to ‘experts’”, which is “an abdication of responsibility”. The questions are too life-changing to leave “solely to people like scientists, innovators, and politicians”. Everyone needs to understand what is coming so we can steer toward “the future we want, rather than one that someone else decides for us.” - What sci-fi is for. Films help “not because they are accurate or prescient, but precisely because they are not tethered to scientific accuracy”. They also have dangers. They cannot invent the undiscovered. They are poor guides to the technology itself. And without “scientific facts and social realism” they leave a “misplaced impression that we’re careering toward a hopelessly dystopian technological future” with nothing to be done. He is warning against fatalism. - “Don’t Panic” (Hitchhiker’s Guide): “we shouldn’t be complacent — far from it”, and there are “deep pitfalls”. Still, “I’m optimistic enough to believe that we have the collective ability” to make technology work for us. Science and technology cannot deliver love and happiness (“you can’t simply ‘science’ your way to them either”), but they can make them easier to achieve. The failure modes to avoid are panic and becoming “so enamored by the tech itself that we become blind to its potential downsides”. - Scope. He contributes to the World Economic Forum’s (WEF) annual Top Ten Emerging Technologies. The book is deliberately incomplete and focuses on “how we think about technological innovation, society, and the future.”

Quotes: - “if we’re tempted to start renouncing technologies from a position of privilege, we risk denying too many people without the same privileges the chance to make their own decisions” - “Yet I fear that this is, in itself, an abdication of responsibility.” - “not because they are accurate or prescient, but precisely because they are not tethered to scientific accuracy”

2019-03-07 · sci-fi-movies-are-the-secret-weapon-that-could-help-silicon-valley-grow-up-a1ebeb91343a#

Title: Sci-fi movies are the secret weapon that could help Silicon Valley grow up Relevance: medium

Provenance: His own prose. It is based on the Conversation article and is a near-duplicate of 2019-01-16.

Argument: Same core as 2019-01-16: the line between could and should (Ian Malcolm), permissionless innovation driven by power and wealth, Ex Machina’s unaccountable genius and machines “that know us better than we know ourselves”, sci-fi as leveler, and the SFIS responsible-innovation community. The differences are small. Ethics “alone are rarely enough. It’s easy for good intentions to get swamped by fiscal pressures and mired in social realities.” Doing better requires “more than good intentions or simply establishing an ethics board”, which anticipates the April piece. The book is “written with innovators in mind”. The applications he lists for the could/should line are gene editing and the CRISPR-babies scandal, de-extinction, human augmentation and AI. It closes: “It certainly beats being blindsided by risks that, with hindsight, could have been avoided.”

Quotes: - “It’s easy for good intentions to get swamped by fiscal pressures and mired in social realities.” - “requires more than good intentions or simply establishing an ethics board”

2019-03-31 · design-principles-for-de-marginalizing-the-future-ae084598edbd#

Title: Design Principles for De-Marginalizing the Future Relevance: medium

Provenance: His own prose. It is the longer version of a short piece he wrote for ASU’s The Guide Project (howtodesignthefuture.asu.edu), published there under the URL slug blessed-are-fairness-seekers.

Argument, in his terms: - The premise. A forward-looking twin of “history is written by the victors” is “the future is designed by the powerful.” - Epistemic humility. “I’m not yet convinced homo sapiens have the tools and abilities to truly design the future”. Science reveals “a fractal-like depth to the complexity of the universe we live in”, and certainty “all too often turns out to be illusion”. Even so, we have a growing toolkit: coding with DNA, the “code” of atoms and molecules in quantum materials, and cyberspace, plus social and institutional design. He also notes our “seemingly-limitless belief in what we can achieve”. - Who decides. The would-be architects of the future are the wealthy, powerful, connected and well-educated. The poor, the homeless, the discriminated-against and the educationally challenged are marginalised, “and yet, designed or not, the future belongs as much to them”. Any design guide must “de-marginalize” the process. Otherwise the default is “a future of the powerful, by the powerful, for the powerful.” - Principles. He secularises the Beatitudes into eight “social design principles”. They include considering the poor, listening to the quiet, esteeming the merciful, elevating people who build bridges, and protecting the persecuted. However powerful our technologies become, they must be channelled “in socially responsible and responsive ways”, toward a future “of the people, for the people, and by the people.”

Notable: He openly draws on a Christian text as a source for design ethics. His doubt about humanity’s ability to design the future comes from complexity thinking.

Quotes: - “the future is designed by the powerful” - “a fractal-like depth to the complexity of the universe we live in” - “a future of the powerful, by the powerful, for the powerful”

2019-04-15 · tech-companies-need-an-ethics-reset-4d936a27960e#

Title: Ethics Boards Won’t Save Big Tech Relevance: high

Provenance: His own prose. It was probably first published at OneZero (Medium): the 2019-06-12 post links to it at onezero.medium.com.

Argument, in his terms: - The case. Google set up an external AI ethics advisory council on 26 March 2019 and disbanded it by 4 April, falling apart “because of ethical challenges”. The larger question is how a company can make sure AI is “as good for society as they are for the company’s bottom line”. - Principles are necessary but insufficient. He calls Google’s AI principles, Microsoft’s approach and the IEEE’s (Institute of Electrical and Electronics Engineers) Ethically Aligned Design “laudable”. But “ethics on their own don’t provide robust mechanisms for developing and building safe and responsible products.” Ethics “are worth little without mechanisms and processes”. - Experience behind the view. Since 2013 he has taught entrepreneurial ethics in the University of Michigan Master of Entrepreneurship. Students had “good intentions but no idea how to translate them into good practice”; their values were “naive and ephemeral”. The course rested on five pillars: principles of entrepreneurial ethics; personal and institutional values; processes for codifying values; engaging key constituencies; and socially responsible practices and products. He came to see the social-risk landscape as a threat to entrepreneurial success (“consequences of social ignorance”, linked to the Risk Innovation Accelerator). He uses threat-to-value language: “every action they took potentially threatened something of importance to someone else”. - His diagnosis. At worst, ethics boards are “smoke-and-mirrors”. At best they set ground rules but change little without ways to put them into practice (he cites The Verge). The AI sector especially has “fallen into the trap of equating ethics with success while skipping over the hard stuff in between”, and treats boards and principles “as a way to blunt criticism”. - His remedy. Focus on outcomes. That means internal and industry-wide standards, measurable expectations, “enforceable checks and balances”, meaningful policies and a culture of responsibility. It needs buy-in across the whole enterprise ecosystem, “from employees and executives to investors, business partners, consumers, affected communities, and regulators”. Most important is training and education over two, five and ten years, and openness to new skill sets and to experts in responsible innovation. The point is not to “throw the ethics baby out with the bathwater” but an “ethics reset”.

Views on AI companies: He is sceptical of AI-sector ethics-washing but not cynical: he assumes good intentions that fail through a gap in implementation. Governance is layered: internal mechanisms first, with regulators and affected communities among the stakeholders.

Quotes: - “ethics on their own don’t provide robust mechanisms for developing and building safe and responsible products” - “At worst, ethics advisory boards can easily become a smoke-and-mirrors attempt to mask business as usual under the guise of social responsibility.” - “fallen into the trap of equating ethics with success while skipping over the hard stuff in between”

2019-04-26 · avengers-endgame-a-cautionary-tale-of-technological-innovation-8cb1c3ca1549#

Title: Avengers: Endgame — a Cautionary Tale of Technological Innovation Relevance: medium

Provenance: His own prose.

Argument, in his terms: - Morality at the movies. The Marvel arc since Iron Man (2008) turns on “the tension between technologically-enhanced power, and social responsibility.” He notes the fight over the word “morality” in his book’s title. - Current capabilities. He lists restarted brain cells in dead pigs, human intelligence genes in monkeys and a 3D-printed heart. These show “the growing tension between what is possible, and how we ensure our technological reach doesn’t exceed our collective grasp.” - The nuclear precedent. “This isn’t a new problem.” The 1945 atomic tests and Oppenheimer’s “destroyer of worlds” are the earlier case, and comic-book heroes are “byproducts of an atomic age”. He uses nuclear conceptually, as the archetype of a power-responsibility tension. The new generation is CRISPR (“redesign whole species, leapfrogging evolution”), planetary geoengineering, and “day-by-day we’re handing more of our lives over to artificial intelligence-based machines.” Such technologies “if left unchecked, could cause immeasurable harm.” - Where the film misleads. In the film, agency rests with “a few exceptional individuals”, and it “naively promotes the idea that, if handled right, powerful tech can be the solution to all our problems.” In reality, “New technologies are developed by large teams of people, and their uses and abuses are tacitly sanctioned by millions more”, and they become “enmeshed in a complex network of societal dynamics”. Navigating them needs “as much social-savvy as they do tech-savvy”. Our resources are “collective intelligence… sense of right and wrong, and our capacity to work together.” - Governance and teaching. The questions are “who decides which new technologies are developed and how they’re used” and how we collectively get “between good intentions and socially acceptable outcomes.” He teaches an ASU undergraduate course, FIS 394 (its URL reads fis-394-moviegoers-guide-future), on sci-fi and responsible innovation.

Quotes: - “how we ensure our technological reach doesn’t exceed our collective grasp” - “we don’t have a band of superheroes to save the day when things go wrong” - “New technologies are developed by large teams of people, and their uses and abuses are tacitly sanctioned by millions more.”

2019-06-12 · want-to-get-smart-about-technology-ethics-these-sci-fi-movies-can-help-3cebedf29c9c#

Title: What can sci-fi movies teach us about technology ethics? Relevance: medium (a listicle, but with compact statements of his positions)

Provenance: His own prose: film blurbs based on Films from the Future. His author line reads “Director of the Arizona State University Risk Innovation Lab”.

Argument and content: “As technologies become ever-more powerful, so do the ethical challenges they raise.” His examples are gene-edited embryos, predictive policing, facial recognition, AI babysitter screening, social-media data privacy and “everything AI”. He recommends 12 films. The points that matter for the map: - Ex Machina. It depicts “a plausible future where AIs learn to press our metaphorical buttons, without being constrained by the cognitive biases and values that limit human behavior”. The AI learned human biases “from studying Google-like searches”. This is the manipulation thread again. - Transcendence. Singularity technologies are “impossible yet entertaining”. The image caption asks whether “myths of a technological singularity” can still spark real-world activism, and he notes how “promises of extreme technological power leads to extreme actions to oppose it”. This shows he is sceptical of the Singularity and alert to the social dynamics it provokes. - Never Let Me Go. Technologies built for the “greater good” can “smash the lives of individuals”, and good intentions “too-easily slip into turning a blind eye to unethical behavior.” - Inferno. It shows “the dangers of slipping into deeply unethical behavior while trying to address perceived global risks”. Its underlying debates are gain-of-function and dual-use research. - Elysium. Inequality of access, and “the way that AI and robots impact rich and poor communities differently.” - Ghost in the Shell. It asks what it means to be human when brain-machine interfaces “erode the boundaries between who you think you are, and who you may be”. This is the being-human thread. - The Man in the White Suit. The socially naive scientist is set in parallel with modern nanotechnology tensions. This is his nano lineage, used conceptually. - Contact. Science and belief, and the discipline of science “elevates who we are.”

Quote: - “AIs learn to press our metaphorical buttons, without being constrained by the cognitive biases and values that limit human behavior”

Low relevance#