The Case for a Global AI Safety Treaty

Diplomatic meetings on artificial intelligence have a peculiar quality. The coffee is always too strong, the rooms are always too cold, and somewhere in the back, a researcher is discreetly telling a minister that borders don’t really matter in math. Over the past two years, I’ve seen a few of these develop, and the difference between what’s being said in public and what’s being whispered in private conversations has only gotten bigger.

The September signing in Vilnius was meant to be a turning point. The Council of Europe’s Framework Convention, the first legally binding international agreement aiming to anchor AI development to human rights and democratic values, was signed by representatives from the EU, the US, the UK, and a few smaller states. The table was packed with photographers. Strong and balanced, according to Marija Pejčinović Burić. However, as you left the building, you could feel how incomplete everything was. The labs developing the most potent systems are not actually bound by the treaty. It ties governments to ideals, which is a completely different animal.

Perhaps more than any clause in the text, that distinction is important. The current generation of frontier AI models is being released into the world more quickly than any oversight regime can keep up with. These models were trained on compute budgets that are close to 10²² floating-point operations. Scholars at the Existential Risk Observatory and organizations such as PauseAI have advocated for compute thresholds that are significantly lower than those set by regulators, with some proposals falling as low as 10²¹ FLOP. The argument is simple. Waiting until something goes wrong is not really a strategy if a model’s capabilities scale with the compute used to train it, and if those capabilities include things like cyberweapon design or biological synthesis assistance.

Skeptics take a strong stance, and they are not entirely incorrect. In Lawfare, Adam Thierer and Keegan McBride argued that international treaties do not truly keep the field safe, but rather American leadership in AI innovation. That makes sense. Ossification of treaties is possible. Countries that do not intend to comply may sign them. Additionally, they have the potential to slow down the very labs conducting the most important safety research. A badly drafted treaty might be worse than none at all, especially if it forces development underground or into countries with laxer regulations.

However, the argument for trying nonetheless continues to grow, and it does so for reasons unrelated to idealism. The supply chain for AI hardware is worldwide. The United States, the Netherlands, Taiwan, South Korea, and a few other nations whose names are hardly ever mentioned in treaty drafts must work together to verify any significant restriction. Whether it’s a deepfake that destabilizes an election or something much worse, damage from a misaligned or abused system won’t end amicably at a border crossing. Frontier labs themselves have begun to admit in their system cards that safety procedures are already being undermined by state-to-state competition.

As this develops, it’s difficult to ignore how the discourse has changed in just two years. Some people dismissed the 2023 London summit as doom-obsessed. By 2025, the language had shifted to emphasize growth, opportunity, and transformative upside. There is truth in both framings. Additionally, both conveniently avoid the more difficult question of exactly who is responsible when a system trained on a trillion parameters performs an unexpected action. That question cannot be addressed solely by a treaty. However, in the absence of one, the answer is usually none.