Plans to fill a gap in the global governance of autonomous weapons are in jeopardy as a group of nations led by Russia and the US move to use a UN conference this month to block proposals to negotiate a legally binding set of rules.
The scale of the opposition will become clear at a diplomatic retreat starting on Friday before the seventh review conference on “certain conventional weapons” in Geneva due to be attended by 128 countries from 16 to 20 November.
The review conference is seen as the key moment when the world could decide whether to back a mandate to negotiate a legally binding instrument negotiated within a defined timetable. The speed with which autonomous weapons are being developed and used in warfare is widely acknowledged.
Anna Hehir, the head of military AI governance at the Future of Life Institute, an independent nonprofit, said: “States have never been closer to launching a global agreement on AI weapons. A lot of the hard work has already been done and states have now agreed by consensus to elements of a draft treaty text that would be the basis for negotiations to start.
“But we will likely see some states hide behind language that has the veneer of progress but advocates for a continuation of endless discussions on voluntary measures that have no teeth. The proposals for more years of discussions on non-legally binding measures are smokescreens to keep autonomous weapons systems unregulated and states unaccountable for their use.”
Discussions have been under way on the issue inside the UN for nearly three years but Russia, the US and Israel, among others, are pushing against the review conference setting a timetable for a legally binding instrument.
In an intervention timed to influence the review conference, Pope Leo XIV stressed that decisions relating to human life could never be given to machines. He said: “Decisions affecting human life, freedom and the use of force must always remain subject to effective, informed and responsible human oversight. For this reason, artificial intelligence must be ‘disarmed’, and I encourage the states meeting in Geneva in November to take concrete steps toward banning lethal autonomous weapons.”
Human Rights Watch and the campaign group Stop Killer Robots argue that weapons systems, assisted by AI, are proven to behave unpredictably, increasing the risk of fatal errors and battlefields spinning out of human control.
The International Committee of the Red Cross (ICRC) said on Thursday: “The time has come to move from discussion to negotiation on new international rules to regulate autonomous systems. The world cannot afford further delays.”
In July, Ukraine reported for the first time that Russia had used an AI-guided drone, which struck a propane tank at a fuel station in Zaporizhzhia. The Ukrainian president, Volodymyr Zelenskyy, has warned about the speed of AI weapon development, calling for the Ukraine war to be ended before AI takes over the battlefield.
On 22 October, a coalition of 50 states including Ireland, Italy, Brazil, Mexico, France and Germany called for the conference to agree a strict negotiating mandate. The ICRC separately reported that more than 70 states supported beginning negotiations on a legal instrument.
The UK did not back the Irish statement and has previously argued that existing international humanitarian law is sufficient basis for the regulation of such weapons.
The Ireland-led group proposed a mandate to be agreed by the review conference that would establish a time-limited expert group tasked with negotiating the needed rules over three sessions in Geneva in 2027, totalling 25 days. The group would be required to submit a report containing the draft protocol to that year’s annual meeting, expected to take place later in 2027.
In September, a UN experts group report prepared for the Geneva review conference tried to define the moral issues at stake by backing general calls for human judgment and control in the use of such weapons, as well as limits on the range and breadth with which they are allowed to operate.
The French government welcomed the experts group proposals, saying they were a sufficient basis for the conference to be given a negotiating mandate that “bans completely autonomous weapons systems and regulates systems with autonomous functions”.
The US largely favours leaving it to nation states to impose their controls in line with international humanitarian law. The Pentagon has set up its own autonomous weapons working group. Similar expert committees in Germany are trying to define the principles behind how human control can be exerted over such weapons.
In August the outgoing UN secretary general, António Guterres, and the ICRC president, Mirjana Spoljaric, warned that the world was “nearing a moral red line where autonomous weapons could kill without human involvement”. They said: “Negotiations must begin now to urgently adopt a legally binding instrument to regulate autonomous weapon systems with clear prohibitions and restrictions.”
A recent poll commissioned by the Future of Life Institute found overwhelming support for a global treaty to limit AI weapons among the public in France, Germany and the UK, but some scepticism that the treaty would ever be respected.
A total of 163 states voted at the UN disarmament and international security committee that they were ready to launch negotiations. They will face enormous pressure to back down and settle for outcomes that keep the issue deadlocked. Multilateral arms control agreements are known to come about when cross-regional majorities withstand pressure and step outside of consensus paralysis.
The expert working group defined autonomous weapons as those weapons “capable of identifying, selecting and engaging a target without the need to refer the issue to a human being”.
The latest example of the misuse of AI in war was the US bombing of a school in Minab, Iran, on 28 February, which killed at least 175 people. The targeting included input from the Maven AI system operated by Palantir. Maven reportedly did not flag that the site was now a school, nor did any personnel tasked with reviewing the AI target data. One advantage of AI is that it is much quicker in checking a target list, but no one picked up that this target had been a school for 10 years.