Lethal Autonomous Weapons Systems—that’s what they are being called within discussions at the UN. We call them Killer Robots.
No one has come up with a legal definition of what they are, partly because there aren’t any yet. But there soon will be. According to an open letter by scientists in the field of machine intelligence, “Artificial intelligence has reached the point where deployment of [autonomous weapons] is feasible in years, not decades. And the stakes are high, autonomous weapons have been described as the third revolution in warfare, after gunpowder and nuclear weapons.”1
Killer robots are machines which will act completely independently, without human supervision. Killer robots select a target and eliminate it. They most likely will not look like Hollywood terminators, but functionally that is the general idea.
Technology engineering is on a curve toward greater automation and also machine intelligence. Technology engineers are developing machines which are learning to think, read, and write, and also to sense the world through the functions we do-the ability to see, hear, touch, taste, and smell.
You likely carry the current result of such technology in your pocket. Your cellphone automatically seeks out cellular signals to stay connected to a communications network, you don’t have to command it to do so. You may have installed apps on your phone which automatically update and do certain functions like notify you when someone posts to their Facebook page. Machines may have assembled part of the phone in an automated process.
This technology is revolutionizing, and will continue to revolutionize, human existence. Our movement has no problems with these scientific advancements if aimed at increasing human well-being. It is the weaponization of this technology that we object to. The military sector is looking to weaponize this technology.
The Campaign to Stop Killer Robots2 is an international civil society movement which is calling for a pre-emptive ban on development and use of autonomous weapons through creation of a new binding international law. We believe there is no way that this technology can be held accountable to the restrictions of international humanitarian law or human rights law.
In April 2016, states party to the Convention on Conventional Weapons met for the third year in a row for informal discussions on autonomous weapons.
The week-long meeting included panels of experts on the concept of machine autonomy, definition of autonomous weapons, humanitarian law, human rights law and ethical issues raised by autonomous weapons. These panels and their presentations are publicly available.3 The panels provided information to government decision-making on the issue. After each panel’s presentations, government representatives of the attending states had the opportunity to question the panel.
Machines at war would function on the underlying software, which would in military machines be secret. Each competing machine system would have its own software which would decide its own response timing and severity.
How would these machines interact once activated? The use of machines in high frequency stock trading could be indicative. Computerized stock trading machines monitor the stock trade and respond according to their own pre- programmed responses. Machines interacting with and responding to other machines are believed to have been a major factor in the 2010 “flash crash” of the US stock market in which a trillion dollars evaporated in 36 minutes. Since that time, “circuit breakers” which halt trading have been put in place to prevent machine trading from going out of control and crashing the market.4
All well and good in a regulated market; however, in a war there is no arbitrator to stop machines which go out of control.
In 2007 an automated gun in South Africa unloaded itself into an onlooking military crowd, killing nine and injuring fourteen. The gun was designed to obtain and feed targeting data from a fire control unit straight to the pair of 35mm guns, and was capable of reloading on its own when its ammunition magazine emptied.
After investigation into the incident the South African military stated that the incident was the result of “an undetected mechanical failure, which the manufacturers of the gun allegedly kept secret.” This Swiss-made weapons system is now possessed by at least a dozen countries.5
In Geneva, the Stop Killer Robots campaign met with the UN Special Rapporteur on Extrajudicial Executions, South Africa’s Chris Heyns, who together with Kenyan lawyer Maina Kia, the UN Special Rapporteur on the rights to freedom of peaceful assembly and of association, have called for a pre-emptive ban on autonomous weapons at the February 2016 session of the Human Rights Council.6 Heyns told the campaign that “without [civil society] activism, there would be no human rights”.
This was a strong endorsement for the important role civil society plays in shaping the international discussion on issues important to humankind, and was acknowledged as such by governments represented at the Geneva talks.
We will need to stay focused to assure that governments don’t step back from making progress towards a ban. So far we have allies in 14 countries which have called for such a ban.7
Machines work well in structured spaces-think of factory floors or of board games like chess or go. War is an unstructured space. Can a machine differentiate between a soldier charging and a refugee fleeing? Its hard enough for a human to do so, and requires relational thinking which is not yet occurring and is not on the horizon.
Scientists have noted that “If any major military power pushes ahead with AI weapon development, a global arms race is virtually inevitable”. They noted that unlike major weapons systems, AI weapons would be inexpensive to mass produce and it would only be a matter of time before they were “in the hands of dictators wishing to better control their populace, warlords wishing to perpetrate ethnic cleansing. Autonomous weapons are ideal for tasks such as assassinations, destabilizing nations, subduing populations and selectively killing a particular ethnic group.”8
We can put a halt on further research and development of autonomous weapons. This has been done before. A global ban on the deployment and use of blinding laser weapons occurred before they had been distributed and used because the weapon violated “the dictates of public conscience.”9
Action point: Call on the Canadian government to support a pre-emptive ban on the development or use of autonomous weapons. Write to: Minister of Foreign Affairs, Stéphane Dion, House of Commons, Ottawa, ON, K1A 0A6 (tel 613 996-5789; fax 613 996-6562; email firstname.lastname@example.org)
Also: support key Canadian organizations in the international campaign to Stop Killer Robots. Canadians made up the largest segment of the international civil society delegation at the April 2016 CCW meetings at the UN in Geneva. Let’s keep it that way!
Dr. Yeshua Moser-Puangsuwan is a member of the International Committee on Robot Arms Control (ICRAC) and an independent scholar on humanitarian disarmament. He works in the research and advocacy division of Mines Action Canada.
1 Autonomous weapons: an Open Letter from A.I. & Robotics Researchers, 28 July 2015.
3 United Nations, 2016 Meeting of Experts on Lethal Autonomous Weapons Systems , 11-15 April 2016
4 The Flash Crash: The Impact of High Frequency Trading on an Electronic Market, report of the U.S. Commodity Futures Trading Commission, 1 October 2010, updated 5 May 2014.
5 Shachman, Noah, “Robot Cannon Kills 9, Wounds 14” Wired Magazine, 18 October 2007.
6 Joint report of the Special Rapporteur on the rights to freedom of peaceful assembly and of association and the Special Rapporteur on extrajudicial, summary or arbitrary executions on the proper management of assemblies, A/HRC/31/66, 4 February 2016.
7 As of April 2016, the following nations are calling for a pre-emptive ban on autonomous weapons: Algeria, Bolivia, Chile, Costa Rica, Cuba, Ecuador, Egypt, Ghana, Holy See, Mexico, Nicaragua, Pakistan, State of Palestine, and Zimbabwe. See updated lists at stopkillerrobots.org.
8 Autonomous weapons: an Open Letter from A.I. & Robotics Researchers, 28 July 2015.
9 Article 1 & 36 of Protocol 1 of the Geneva Conventions prohibits governments from acquiring a new weapon, means or methods of warfare prohibited by International Humanitarian Law or which violates ‘the dictates of public conscience’.