Killer robots: it’s not the AI that’s the problem

In a recent open letter, Tesla’s Elon Musk and others called for a ban on autonomous weapons, saying “Lethal autonomous weapons threaten to become the third revolution in warfare. Once developed, they will permit armed conflict to be fought at a scale greater than ever, and at timescales faster than humans can comprehend. These can be weapons of terror, weapons that despots and terrorists use against innocent populations, and weapons hacked to behave in undesirable ways. We do not have long to act. Once this Pandora’s box is opened, it will be hard to close.

Yet autonomous weapons are already with us, after a fashion. And artificial intelligence isn’t actually the biggest problem.

A bullet, during the second or so that it is in flight, autonomously follows the laws of physics. But the world is not likely to have changed much during that time. If shooting the bullet was appropriate, that will still be true when it hits. A cruise missile can fly for several hours, and home in on a precise spot, specified by GPS coordinates – although things may have changed during those hours of flight.

Smarter again is a heat-seeking or radar-guided missile, which can home in on an aircraft, even one doing it’s best to evade the threat – yet it cannot distinguish passenger aircraft from military aircraft. The next step up are systems guided by IFF, which can distinguish friend from foe. After that comes the kind of AI that Elon Musk is talking about.

The ultimate extreme is the “Menschenjäger” of Cordwainer Smith’s 1957 short story “Mark Elf.” The Menschenjägers were built by the “Sixth German Reich” to seek out and kill their non-German enemies (whom they could infallibly detect by their non-German thoughts). Being virtually indestructible, the last Menschenjäger had travelled around the planet on this mission 2328 times by the time the story is set. Since no Germans were alive at that point, there was nobody left to shut it down.

The real problem with the Menschenjägers was not their AI, but their persistence in time. A similar problem arises with that most stupid of autonomous weapons, the landmine. Sown in their tens of millions, landmines continue to kill and maim for decades after the war that buried them is over.

It isn’t really a matter of whether the weapon has AI or not – it’s whether the weapon has an off switch or a self-destruct mechanism. No weapon should keep on pointlessly killing people.

Network Topology in Command and Control

I was happy to get my copy of Network Topology in Command and Control: Organization, Operation, and Evolution in the mail today – particularly because it includes my chapter “C2, Networks, and Self-Synchronization” (which rounds off several aspects of my work in this space). There are other interesting chapters too, of course!

Fields of Conflict: a book review

Fields of Conflict, edited by Douglas Scott, Lawrence Babits, and Charles Haecker

I recently read Fields of Conflict: Battlefield Archaeology from the Roman Empire to the Korean War, edited by Douglas Scott, Lawrence Babits, and Charles Haecker. Having twice visited the site of the Battle of Gettysburg (150 years ago today!), this book caught my eye.

The Gettysburg Battlefield, site of the battle on 1–3 July, 1863

This collection of articles gives a fascinating account of conflict archaeology or battlefield archaeology. Essentially, this discipline treats ancient battlefields as crime scenes, and applies modern investigative techniques. There is a discussion of the site of the Battle of the Teutoburg Forest, for example (to quote another book: “To lose one legion, Mr Varus, may be regarded as a misfortune; to lose three looks like carelessness”). An amateur archaeologist located the long-lost site of this battle in 1987, by finding Roman coins and sling-shot ammunition. Later investigation showed that many Romans died just in front of an improvised (though extensive) defensive wall constructed by the Germanic tribesmen. Collapse of the wall may have helped preserve the artefacts.

Reconstructed Germanic fortification from the Battle of the Teutoburg Forest (photo: Markus Schweiß)

Such finds provide a way of assessing the accuracy of (and in some cases, re-evaluating) contemporary written accounts, and of better understanding what precisely took place. For example, GIS analysis of ammunition finds can be used to infer where cannon were located

Inferring the location of cannon (photo:

The book includes interesting discussions of the sites of the Battle of Towton (1461), the Battle of Zboriv (1649), the Battle of Fallen Timbers (1794), the Battle of Mackinac Island (1814), the Battle of Lookout Mountain (1863), the Battle of Hembrillo Basin (1880), and several others. There is also an article on the archaeological investigation of Stalag Luft III (famous for the “Great Escape” in 1944).

The book gives many cases where metal detectors have found artefacts such as buttons, coins, lance-heads, crossbow bolts, or these bullets from the US Civil War

A interesting point not raised in the book is that the process of battlefield archaeology can itself be the subject of formal analysis, as discussed in the article “Simulating archaeologists? Using agent-based modelling to improve battlefield excavations” by Xavier Rubio Campilloa, Jose María Celaa, and Francesc Xavier Hernàndez Cardona. The book does note the impact of amateur collecting on battlefield sites, and agent-based modelling may also provide a way of formally studying that.

As the book points out, an understanding of terrain and of lines of sight is essential in battlefield archaeology (Lookout Mountain, Tennessee, 1864)

This book is an excellent introduction to the field (the articles in the book are derived from papers presented at the third Fields of Conflict Conference (a continuing conference series, with the seventh held last year). For related reading, see:

Fields of Conflict: 3.5 stars