Late Lessons, Jensen Huang and AI

LL1-17 digest: Ch17 Conclusions#

Late lessons from early warnings: the precautionary principle 1896–2000, EEA 2001. Report and PDF pages 192–194. No byline. It is most plausibly the editorial team’s collective conclusion (Harremoës, chair; Gee, MacGarvin, Stirling, Keys, Wynne, Guedes Vaz); Ch16, which it summarises, is credited to the “Editorial team”. Who drafted it is not recorded. No panels.

Core argument#

The chapter compresses the report into a framework and a list.

Two observations from the 14 cases (p. 192): 1. Regulating technology means balancing the costs of being too restrictive against those of being too permissive, under uncertainty and ignorance. Some of the harms that followed inaction could not have been foreseen. 2. Warnings, even “loud and late” ones, were clearly ignored. Appraisal was too narrow. Some regulatory actions were taken without weighing alternatives or how they would work in practice.

Definition. The precautionary principle is defined broadly, as “an overarching framework of thinking that governs the use of foresight” under uncertainty and ignorance, when both action and inaction may be costly (p. 192). This is consistent with Ch1, which says precaution requires “much more than” a trigger for action (p. 13).

Table 17.1 (p. 192) separates three states of knowledge and pairs each with a type of action:

State of knowledge Example and dates Action
Risk asbestos, from 1965 prevention
Uncertainty antibiotics in animal feed, from 1969 precautionary prevention
Ignorance CFCs before 1974; mesothelioma before 1959 precaution

The “precaution” row does not depend on knowing the specific harm. It screens on persistence and bioaccumulation, casts the information net wide, monitors over the long term, and keeps technologies diverse, with fewer “monopolies”.

Pages 192–193 then argue for: - procedures that are fair, transparent and accountable, to restore trust (pp. 192–193); - integrating health and environment; - widening appraisal to include social issues, with stakeholders involved from the earliest stage (said to add information and possibly trust, without necessarily stifling innovation); - recognising that current controversies (GMOs, oil-rig disposal) turn on values; - humbler science that can handle complexity.

Level of proof (p. 193). The key analytic move is that the level of proof needed for action, anywhere from the EC’s “reasonable grounds for concern” to “beyond reasonable doubt”, is a political and ethical choice. It shifts the size, nature and distribution of the costs of being wrong. It should depend on the harm, the claimed benefits, the alternatives, and the costs of error in both directions. (Ch16’s own ladder goes lower still, to “scientific suspicion of risk”, Table 16.1, p. 184; Ch17 leaves that rung out.)

§17.1 (pp. 193–194) lists the twelve lessons: respond to ignorance; monitor over the long term; find blind spots; reduce interdisciplinary obstacles; account for real-world conditions; scrutinise claimed benefits; evaluate alternatives and promote robust, diverse technologies; use lay and local knowledge; account for different groups’ values; keep regulators independent of interested parties; reduce institutional obstacles; avoid “paralysis by analysis” by acting on reasonable grounds for concern.

Closing (p. 194). Most lessons are about better information. None removes the underlying dilemmas, but they “would at least increase the chances” of anticipating costly impacts, balancing pros and cons, and minimising the costs of surprises. Precaution “can also” stimulate innovation (through diversity and flexibility) and better science. Over-precaution “can also be expensive”, through lost innovation and lost lines of inquiry. The central conclusion is that maximising innovation while minimising hazards is “ultimately a matter of political discourse” and could be done better by embracing the twelve lessons.

Key evidence#

Ch17 contains almost none of its own; it summarises Ch16. The dates in Table 17.1 match the case tables in part: - Antimicrobials: 1969 is the Swann Committee report (the committee was set up in 1968, Ch9, p. 94; Ch16 also has an inconsistent “1967” and “early 1968”, pp. 172, 174). Swann was partly implemented in the 1970s, then relaxed from 1975 (Table 9.2, p. 99). - CFCs: 1974 is Molina and Rowland. The report treats pre-1974 CFCs as a strong case of genuine ignorance (Ch16, p. 169). The earlier laboratory work on chlorine and ozone (1907, 1934) and Lovelock’s 1973 survey (Table 7.1, p. 83) give the objection some weight but do not overturn that reading. - Mesothelioma: 1959 matches its identification in South Africa in 1959–60. Ch5 (p. 55) notes scattered associations in the 1940s–50s. The table leaves 1959–65 unclassified. - Asbestos as a known risk: 1965 marks consensus on mesothelioma (Ch5, pp. 55, 58). As a date for “respiratory disease, lung and mesothelioma cancer” together, it compresses a longer history: the report’s own Table 5.1 (p. 61) records asbestosis in 1930 and lung cancer in 1955.

Main mechanisms#

Transferable insights (technology-neutral)#

  1. Distinguish risk, uncertainty and ignorance, and date them (p. 192). Strong as a concept; moderate as a cause of the historical failures.
  2. Act on ignorance with strategies that don’t depend on knowing the specific harm: property screening (persistence, bioaccumulation), attention to irreversibility and scale, monitoring, diversity (p. 192; lesson 7, p. 194; Ch16, pp. 170–171). Moderate.
  3. The threshold of proof is a political choice about who bears the cost of error (p. 193). Strong.
  4. Warnings are often available but unused (p. 192). Moderate to strong within the cases; no base rate.
  5. Interventions adopted without appraising alternatives can create new hazards (p. 192). Moderate.
  6. Silos produce blind spots (p. 193; lessons 3–4). Moderate.
  7. Scrutinise claimed benefits as hard as risks (lesson 6, p. 193). Moderate.
  8. Monocultures magnify surprise; diversity is insurance (p. 192). Suggestive.
  9. Independence from interested parties, and institutional capacity to learn, strongly affect how quickly a system responds (lessons 10–11, p. 194). Moderate (the evidence is in Ch16, pp. 178–181, which notes that not every case shows distortion from non-independent information).
  10. Controversies turn on values; early inclusion may build trust (p. 193). Suggestive / asserted.
  11. Precaution can redirect rather than simply stop innovation (p. 194). Asserted. Ch16 concedes that some pathways may need to be curtailed or ended (p. 182).
  12. Better information does not dissolve the dilemma; decisions stay political (p. 194). Strong.

Main caveats#