How Trump and Xi Can Do AI Safety

As the US and Chinese leaders prepare to meet again, there is still little concrete progress on the AI safety dialogue announced in May. Past dialogues have often devolved into mutual recrimination or been derailed by unrelated events. The author, a former US diplomat, argues that the key is to start talking, but to choose topics wisely.

Lessons from past failures

Historical US-China dialogues have repeatedly been disrupted: after Pelosi's visit to Taiwan, China suspended several dialogues; during the spy balloon incident, the PLA did not even answer the US defense secretary's call. Paper agreements are hard to enforce, as seen with the 2015 pledge on cyber theft, which proved unverifiable.

Experts recommend starting small

The author surveyed two groups: former officials involved in official dialogues and researchers in Track II AI safety talks. They rated twelve possible topics for cooperation. Strikingly, both groups agreed on the value of cooperation, but differed sharply on feasibility. Former officials were pessimistic about most topics, considering only nuclear risk and open-source software vulnerabilities as likely to succeed.

What is worth discussing

Combining feasibility and value, experts suggested prioritizing: preventing AI from enabling chemical, biological, or nuclear threats, discussing biosecurity controls, and addressing risks from non-state actors. These areas are technical, with aligned interests, and can start with low-cost information sharing and escalate to genuine cooperation.

Don't forget to involve experts

Governments should actively solicit input from AI labs, academia, and experts, using public comment periods to fill information gaps. For instance, ask industry: which safety measures would be less effective without the other country's participation? How can we detect risk signals from open-weight models?

The hawk's critique

Hawks argue China will not concede in talks, may be stalling, or using them for propaganda as a 'responsible power.' Both sides may define terms like 'safe' and 'controllable' differently, making agreements weak. The author concedes these points but argues they do not mean cooperation is futile. Export controls are already discussed elsewhere; on values, the US should hedge by building its own capabilities.

Start doing

The author suggests the September summit should at least affirm 'do no harm' and show follow-through on the May announcement, providing political cover for practical engagement. Grand treaties are unlikely in the near term, but robust information sharing on issues of mutual interest can build a foundation for more ambitious arrangements. Given the fast pace of AI development, governments need to act quickly to prevent catastrophic risks from becoming reality.