Catastrophic Risks: A Plan for Nuclear Weapons, Engineered Pathogens and Advanced AI, Seen from September 2026

Introduction

Most of the problems on this site cause harm every day and can be counted in deaths per year. Nuclear war, a deliberately engineered pandemic or a failure to keep control of very capable artificial intelligence might never happen in a given year, but any one of them could kill hundreds of millions or billions of people in a short time, and some outcomes could not be reversed. Even a modest annual probability of damage on that scale justifies serious effort.

Each of the three threats depends on a technology that relatively few actors control today, the rules meant to govern it are weaker than the technology, and the cost of misuse is falling as the tools spread. A nuclear exchange would also cause a global famine, and a pandemic undoes years of progress against poverty, so these risks bear directly on the other problems on this site.

This essay sets out a plan for reducing all three risks and then tests it against events of 2025 and 2026. Arms control between the nuclear powers, the expiry of New START and the new weapons in orbit are covered in the companion essay Ending War: A Plan Tested Against the World of September 2026; this essay refers to that material without repeating it and concentrates on what the war plan does not cover.

Where things stand

Nuclear weapons. At the start of 2026 the nine nuclear-armed states held about 12,187 warheads. About 9,745 were in military stockpiles for potential use, about 4,012 were deployed with missiles and aircraft, and between 2,100 and 2,200 were kept on high alert on ballistic missiles, mostly by Russia and the United States (SIPRI). China has about 620 warheads and is expanding its arsenal faster than any other country (SIPRI). The world total fell for decades as Russia and the United States dismantled old warheads, but SIPRI now expects it to rise because dismantlement is slowing while new deployments speed up (SIPRI).

The direct death toll of a large exchange would be only part of the harm. A 2022 study in Nature Food modelled the soot from burning cities and its effect on crops, fisheries and livestock. It found that more than 2 billion people could die from a nuclear war between India and Pakistan, and more than 5 billion from a war between the United States and Russia, mostly from famine (Xia et al., Nature Food, via NASA GISS). A US National Academies report commissioned by Congress and released in June 2025 found major uncertainties and data gaps at every stage of the chain from detonation to environmental effect, and it called for coordinated modelling work to narrow them (National Academies).

Engineered pathogens. The Biological Weapons Convention, the main treaty banning these weapons, has no verification system. Its Implementation Support Unit was set up in 2006 with three full-time staff and has had four since a temporary post was added for 2023 to 2027 (UNODA). In December 2024, 38 scientists writing in Science warned that “mirror” bacteria, built from molecules with the reverse orientation of natural life, could evade immune systems and should not be created (Wikipedia: Mirror life). The Global Preparedness Monitoring Board reported in May 2026 that outbreaks are becoming more frequent and more damaging (GPMB).

Advanced AI. The International AI Safety Report 2026, written by more than 100 experts under Yoshua Bengio, found that general-purpose AI systems have reached gold-medal level on International Mathematical Olympiad problems and that AI agents can reliably complete some programming tasks that take a human about half an hour (International AI Safety Report). The report found that current systems can provide information about biological and chemical weapons development, including expert-level laboratory instructions, and that several developers released models in 2025 with extra safeguards because testing could not rule out that the models would help novices build such weapons (International AI Safety Report). It judged that current systems do not pose a risk of loss of human control, while warning that models increasingly tell test settings apart from real use. Researchers disagree widely about the longer-term danger; in a 2023 survey of 2,778 AI researchers, the median respondent put a 5 percent probability on outcomes as bad as human extinction (Grace et al., AI Impacts).

Part I: The plan

The first two elements of the plan deal with nuclear weapons, the next three with biology and the next two with AI; the last covers funding. Treaty limits on nuclear forces, crisis hotlines and the rules for space are part of the war plan and are not repeated here.

1. Lengthen nuclear decision time and keep humans in charge

A standing nuclear danger is a launch caused by a false warning or a misread crisis, with leaders given minutes to decide. The plan asks the nuclear powers to take missiles off high alert where they can, to adopt launch procedures that allow more time for verification, and to state formally that no automated system will make or recommend a launch decision without human judgment. The main obstacle is the belief that a slower posture invites a first strike.

In November 2024 the presidents of the United States and China affirmed “the need to maintain human control over the decision to use nuclear weapons” (White House archive). The plan seeks a joint statement among all five permanent members of the Security Council, followed by technical exchanges on how each keeps AI out of launch decisions.

2. Keep the testing moratorium and prepare for the worst case

The informal moratorium on explosive nuclear testing limits the development of new warhead designs and removes a visible signal of escalation. The plan asks every nuclear power to state that it will not test, to support the international monitoring network of the Comprehensive Nuclear-Test-Ban Treaty Organization, and to settle allegations of secret testing through technical exchanges and data sharing.

The plan also asks governments to treat a sudden, multi-year loss of sunlight and harvests, whether after a nuclear war or a very large volcanic eruption, as a planning scenario. It funds the modelling work the National Academies recommended and asks major food-producing countries to include a global crop failure of that kind in their food-security planning. That work overlaps with the grain and fertilizer reserves proposed in the starvation essay.

3. Screen DNA orders and govern the riskiest experiments

Anyone trying to build a dangerous pathogen from scratch has to obtain its genetic material, and most of that comes from commercial DNA synthesis companies. Screening every order against a list of sequences of concern, and checking who the customer is, is one of the cheapest controls available. Free, open-source screening software already exists (IBBIS). The United Kingdom published screening guidance for providers and users of synthetic DNA, including benchtop synthesisers, in October 2024 (GOV.UK). The plan makes screening a legal requirement for all providers and benchtop machines in major markets, with a shared international standard so that customers cannot move to an unscreened supplier.

Research that deliberately makes pathogens more transmissible or more deadly needs a separate and stricter process: independent review before funding, disclosure of what is being done and where, and the same rules for privately funded laboratories as for publicly funded ones. For mirror life, the plan supports an international agreement not to create mirror organisms, including the limit on the size of mirror DNA that the UK Government Office for Science has proposed (GOV.UK).

4. Give the Biological Weapons Convention a working staff and a science review

The treaty that bans biological weapons is administered by four people. The plan asks states parties to expand the Implementation Support Unit and to adopt the three mechanisms their own working group has drafted: an international cooperation and assistance mechanism, a science and technology review mechanism to track developments such as AI-aided design, and a working group on compliance and verification (UNODA). Full verification is unlikely soon because biological work is small, dual-use and easily hidden, but regular scientific review and voluntary transparency visits are achievable now.

5. Detect new pathogens early and make vaccines fast

If prevention fails, the speed of detection and response largely determines the death toll. The plan funds genomic surveillance of wastewater, airports and hospitals that can spot a new pathogen before it spreads widely, whether it is natural or engineered. It backs the CEPI goal of having a safe, effective vaccine ready within 100 days of identifying a new pandemic threat; CEPI cites an estimate that about 8.3 million COVID-19 deaths could have been averted by the end of 2021 with faster vaccines (CEPI). It also asks governments to finish and ratify the WHO Pandemic Agreement, which member states adopted in May 2025 by 124 votes to none, with 11 abstentions; an annex still under negotiation is to set rules for sharing pathogen samples and the benefits that come from them (UN News).

6. Test frontier AI before release and report what is found

The companies building the most capable AI systems should test them for dangerous capabilities before release, publish their safety frameworks, report serious incidents to government, and give independent government evaluators access to their models. Parts of this are already common practice: twenty companies signed the Frontier AI Safety Commitments made at the Seoul summit in 2024, which included publishing safety frameworks (GOV.UK), and in 2025 two leading developers applied stricter safeguards to new models because they could not rule out that the models would give meaningful help with biological weapons (Anthropic; OpenAI). The case for making the practice mandatory rests on evidence that voluntary commitments are uneven. An independent index published in July 2026 gave no major developer a grade above C+ and found that “safety rhetoric outpaces revealed behavior” (Future of Life Institute).

Model safeguards and the DNA screening in element 3 form two layers of the same defence against the most concrete near-term AI danger, help with biological weapons. On one benchmark, the Virology Capabilities Test, an AI model scored 43.8 percent against an average of 22.1 percent for expert virologists answering questions in their own specialties (SecureBio).

7. Build shared scientific understanding and channels between rivals

Because governments cannot agree on rules for a technology whose risks they assess differently, the plan supports an international scientific body, on the model of the Intergovernmental Panel on Climate Change, that reports regularly on AI capabilities and risks, and a standing dialogue in which governments discuss the findings. It also asks the United States and China, which build most frontier systems, to hold regular technical talks on AI safety, including shared evaluation methods and the military uses of AI, as they have agreed on human control of nuclear launch decisions.

8. Pay for the work

Private funding for nuclear risk reduction has shrunk. Between 2014 and 2020 the MacArthur Foundation’s nuclear programme made 228 grants worth $100.8 million (MacArthur Foundation), and the foundation left nuclear grant-making after its final grants in 2023 (MacArthur Foundation). The plan asks governments and foundations to fund arms-control expertise, biosecurity staff at the Biological Weapons Convention and the WHO, and independent AI evaluation, at sums far below the missile-defence budgets described in the war essay.

Sequencing

The first two years should go to steps that are cheap, reversible and within existing authority: mandatory DNA screening, pre-release AI testing, a five-power statement on human control of nuclear launches and continued observance of the testing moratorium. Over two to ten years the aim is treaty-level work: a stronger Biological Weapons Convention with a science review, a ratified pandemic agreement, and international AI evaluation standards. Over the longer term the goal is a verification system for biological weapons and binding rules for the most capable AI systems, both of which depend on trust built in the earlier stages.

What makes it stick politically

Most elements serve each country’s own security, and measures framed that way, backed by defence and intelligence agencies, tend to survive changes of government better than measures framed as concessions. DNA providers and AI developers that already screen and test have an interest in rules that make competitors do the same. Published test results and incident reports give legislatures and the public a way to check that commitments are kept.

Where reasonable people disagree

People disagree about how large and how near these risks are. Estimates of AI risk range from negligible to very high, and some critics argue that attention to speculative harms draws money from present ones such as fraud and job loss; others reply that the worst outcomes are large enough to justify spending even at small probabilities.

Supporters of rapid AI development argue that regulation will hand the lead to less careful rivals, especially China; supporters of regulation argue that a race without safety testing raises the risk for everyone, including the winner.

Publishing research, pathogen data and AI model weights helps science and allows independent scrutiny, but it also helps people who would misuse them. Deterrence theorists argue that some nuclear weapons on high alert make war less likely by making a first strike pointless, while critics see high alert as the main source of accidental risk.

Part II: The plan in the world of September 2026

Nuclear: the moratorium under pressure and the treaty review failed

In October 2025 President Trump said he had instructed the military to begin testing nuclear weapons “on an equal basis” with other countries. Energy Secretary Chris Wright said days later that the planned tests were “not nuclear explosions” but non-critical explosions of weapon components (Fortune/AP). President Putin responded by ordering officials to prepare proposals for possible nuclear test preparations, while saying Russia would test only if the United States did first (CSIS).

In February 2026 a US under secretary told the Conference on Disarmament that China had conducted a small, concealed yield-producing test on 22 June 2020 near Lop Nur. China called the claim groundless. The head of the test-ban treaty organisation said it was not possible to assess the cause of the seismic events with confidence (OPB/NPR; Bulletin of the Atomic Scientists). No explosive nuclear test by any country was reported in 2025 or 2026, so the moratorium has held, but an allegation that cannot be settled by evidence invites a matching test, which is why the plan proposes technical exchanges.

The Review Conference of the Non-Proliferation Treaty, held in New York from late April to late May 2026, ended without a consensus final document, the third failure in a row after 2015 and 2022 (UN News). Talks broke down over language singling out Iran, against a background of disputes over the US and Israeli strikes on Iran, the war in Ukraine and the Zaporizhzhia nuclear plant, and the end of New START (ETH Zurich CSS). The next review conference is in 2031. The expiry of New START on 5 February 2026 (Federation of American Scientists) and the new US weapons in orbit are covered in the war essay.

In a contrary direction, the Treaty on the Prohibition of Nuclear Weapons has 75 states parties, with Tonga acceding in July 2026 and Gabon signing in September (UN Treaty Collection). On element 1, no new five-power or US-China statement on human control of nuclear launch decisions could be found for 2025 or 2026, so the 2024 affirmation remains the high point.

The Iran war, which began on 28 February 2026, adds a proliferation question the plan must now absorb: what becomes of Iran’s enriched uranium while the country is at war. The war essay covers the diplomacy.

Biosecurity: the rules are being rewritten while the global system weakens

The United States completed its withdrawal from the WHO on 22 January 2026, ending all funding and withdrawing all staff (CDC). The WHO said the withdrawal made “both the United States and the world less safe” (WHO). Argentina’s withdrawal took effect on 17 March 2026 (Al Jazeera). The withdrawals come in the same period as the aid cuts described in the starvation and poverty essays.

The Pandemic Agreement adopted in 2025 cannot open for signature until member states agree an annex on sharing pathogens and the benefits derived from them. Negotiators missed the May 2026 deadline, and the World Health Assembly extended talks to May 2027 at the latest (WHO). A July round ended with “differences remain” (WHO), and the September round made little progress, leaving the treaty in limbo (NPR via GPB). Element 5 is therefore stalled at the international level.

US research oversight has tightened on experiments but not on DNA purchases. An executive order in May 2025 paused federally funded dangerous gain-of-function research and directed agencies to revise or replace the 2024 federal framework for screening DNA synthesis orders (ASPR). A new policy issued on 20 July 2026 bars federal funding for such research, restricts research with designated foreign institutions, and extends monitoring to non-federally funded work (Butler Snow). The replacement screening framework has not appeared; the federal page on synthesis screening still says it will be updated once a revised framework is available (ASPR). The new policy governs what funded laboratories may study but leaves open the cheaper control on who can buy dangerous sequences, which element 3 treats as the first priority.

In the Biological Weapons Convention working group, states parties have drafted the three mechanisms in element 4 and a proposal to expand the support unit from four staff to seven, though the numbers remain in brackets (UNODA). The ninth session ended on 21 August 2026 with only a procedural report, though the support unit counted about 93.5 percent of the text as agreed or nearly agreed. A final session in December has five working days before the mandate runs out, and Italy’s ambassador warned that the process “risks imploding” (BWPP).

AI has become part of the biological threat. Developers added safeguards to new models in 2025 because they could not exclude that the models would help novices with biological weapons (International AI Safety Report), and in January 2026 the Bulletin of the Atomic Scientists listed AI-aided pathogen design among its reasons for setting its Doomsday Clock at 85 seconds to midnight, the closest it has ever been (Bulletin of the Atomic Scientists). That makes DNA screening more urgent, since it is the control that works regardless of where a would-be attacker gets the design.

AI: more science, less agreement on rules

Of the AI elements, international scientific assessment has advanced furthest. The UN General Assembly created a 40-member Independent International Scientific Panel on AI and a Global Dialogue on AI Governance in August 2025. The panel, co-chaired by Bengio and Maria Ressa, issued a preliminary report on 1 July 2026, and the first Global Dialogue met in Geneva on 6 and 7 July. Bengio told the meeting that “science currently cannot guarantee that as capabilities continue to increase, AI will not cause catastrophic harm” (UN News). No negotiated outcome document was reported, and the next session is set for May 2027 in New York (UN Geneva).

Binding rules have moved more slowly: the EU AI Act’s obligations for general-purpose models have applied since August 2025, and the EU AI Office gained full enforcement powers in August 2026, but the rules for high-risk systems have been postponed to December 2027 and August 2028 (European Commission; Gibson Dunn).

In the United States, the July 2025 AI Action Plan set out more than 90 federal actions focused on innovation, infrastructure and diplomacy (White House), and a December 2025 executive order created a Justice Department task force to challenge state AI laws and asked Congress for a national standard that would preempt them (White House). Several states have moved toward regulation instead. California’s Transparency in Frontier Artificial Intelligence Act, signed in September 2025, requires the largest developers to publish safety frameworks, report incidents and protect whistleblowers (Office of the Governor of California), and New York’s RAISE Act, signed in December 2025, adds 72-hour incident reporting and takes effect in January 2027 (Office of the Governor of New York; Wiley). A bipartisan draft in the House in June 2026 proposed a three-year federal preemption of state laws on model development, but it has not been enacted (Roll Call). The federal Center for AI Standards and Innovation tests models for cyber, biological and chemical risks (NIST), but developers have no federal legal duty to submit to it, so element 6 exists only in patches.

The limits on military AI became the subject of litigation. In March 2026 the Department of Defense designated the AI developer Anthropic a supply-chain risk after the company refused to drop restrictions on mass domestic surveillance and on autonomous weapons without human targeting decisions (TechCrunch). A federal district judge in California held one designation unlawful in August (TechCrunch), and on 25 September a divided appeals court in Washington upheld a second designation made under a different statute (Defense News). The dispute shows that the limits on military AI that element 1 and the war essay call for are being settled in procurement fights and courtrooms and have not been set by statute or treaty.

The US-China channel in element 7 remains thin, and the two countries deal with each other on AI mainly through trade policy. China’s July 2025 Global AI Governance Action Plan supports risk assessment, safety frameworks and the UN mechanisms (Ministry of Foreign Affairs of China). In January 2026 the United States moved to case-by-case licensing for exports of advanced Nvidia H200 and AMD MI325X chips to China (Bureau of Industry and Security), and placed a 25 percent tariff on those chips (White House). No regular bilateral AI safety talks could be confirmed for 2025 or 2026.

Part III: Revised priorities

Save the Biological Weapons Convention working group in December. Adopting the science review mechanism and the expanded support unit, even with verification deferred, is the most time-sensitive step available; if the mandate lapses, years of drafting are lost.
Require DNA synthesis screening. The United States should publish the replacement framework its own executive order called for, with enforcement, and other major markets should adopt compatible rules. AI-aided design makes this the most urgent biological control.
Keep the test moratorium and settle the China allegation technically. Nuclear powers should reaffirm that they will not conduct explosive tests, and Washington and Beijing should use data exchanges or CTBTO consultations to address the June 2020 event.
Seek a five-power statement on human control of nuclear launch decisions, building on the 2024 US-China affirmation.
Finish the pandemic agreement annex before May 2027, so that the treaty can open for signature, and replace the preparedness work the WHO has lost with funding from other members.
Put pre-release testing and incident reporting for frontier AI on a legal footing. If Congress preempts state laws, the federal standard should carry forward the testing, reporting and whistleblower protections already enacted in California and New York.
Open a US-China technical channel on AI safety and military AI, using the UN scientific panel’s reports as common ground.

Conclusion

The events of 2025 and 2026 show movement in opposite directions. Scientific understanding improved, since the International AI Safety Report, the UN scientific panel and the National Academies nuclear study give governments better evidence than they had two years ago, while the institutions that turn evidence into rules weakened. The Non-Proliferation Treaty review failed again, New START lapsed, the United States left the WHO, the pandemic agreement stalled and the Biological Weapons Convention working group is close to running out of time.

Cheap controls that each country can adopt on its own, such as DNA screening, a testing moratorium and human control of launch decisions, now matter more because the multilateral routes are blocked, and none of them requires the great powers to trust one another. AI-aided design has moved biological risk nearer the top of the list, which is why the revised priorities put the Biological Weapons Convention talks and DNA screening first.

This essay reflects reporting available as of 28 September 2026. Several of the processes described, including the Biological Weapons Convention talks and the court case over military AI, may change within weeks.