Autonomous Ragnarok

There are two stories that caught my attention today. In the first there was another recent Tesla death, a fatality up in Fontana. Apparently the car ran into an overturned semi-truck. There have been a spate of autonomous vehicle accidents of late.

Since Tesla introduced Autopilot in 2015, there have been at least 11 deaths in 9 crashes in the United States that involved Autopilot. Internationally, there have been at least another 9 deaths in 7 additional crashes.

This comes on the heels of the story of the Einstein in the Bay Area who refuses to sit in the front seat of his Tesla and keeps getting arrested.

The other story was about autonomous warfare over at Wired, The Pentagon inches towards letting AI control weapons.

General John Murray of the US Army Futures Command told an audience at the US Military Academy last month that swarms of robots will force military planners, policymakers, and society to think about whether a person should make every decision about using lethal force in new autonomous systems. Murray asked: “Is it within a human's ability to pick out which ones have to be engaged” and then make 100 individual decisions? “Is it even necessary to have a human in the loop?” he added.

At the same event, Lieutenant General Clinton Hinote, deputy chief of staff for strategy, integration, and requirements at the Pentagon, says that whether a person can be removed from the loop of a lethal autonomous system is “one of the most interesting debates that is coming, [and] has not been settled yet.”

Sounds great but will it be any more foolproof than the autonomous cars? Will the brass and the bean counters figure out what they consider is an acceptable rate of innocent deaths and collateral damage when the robot driven bullets and bombs start to fly and the wrong target gets wasted?

I'm afraid that the answer will be yes. We have already set up such a metric in dealing with innocent middle east bomb casualties.

You can create the finest machines in the world and they will still lack one key ingredient, in short supply as it is, human judgment. I shudder to think of an autonomously weaponized future.