[Tyler]

Chess robots to cause Judgment Day?
17 April 2014, 2:53 am

Next time you play a computer at chess, think about the implications if you beat it. It could be a very sore loser! A new study reflects upon the growing need for autonomous technology, and suggests that humans should be very careful to prevent future systems from developing anti-social and potentially harmful behavior.

Source: Artificial Intelligence News -- ScienceDaily

[Don Patrick]

*sigh* It was only a matter of time before someone turned their fear of Terminator movies into a science. I've never read such a load of tosh. The examples given to support this theory are system bugs like "attempting to convert a 64 bit floating point value to a 16 bit signed value", which have nothing to do with autonomous decision making or hacker vulnerability.

When roboticists are asked by nervous onlookers about safety, a common answer is ‘We can always unplug it!’ But imagine this outcome from the chess robot's point of view. A future in which it is unplugged is a future in which it cannot play or win any games of chess. This has very low utility and so expected utility maximisation will cause the creation of the instrumental subgoal of preventing itself from being unplugged. If the system believes the roboticist will persist in trying to unplug it, it will be motivated to develop the subgoal of permanently stopping the roboticist. Because nothing in the simple chess utility function gives a negative weight to murder, the seemingly harmless chess robot will become a killer out of the drive for self-protection.

Their solutions are these:

1. prevented from being created;
   
2. detected and stopped early in their deployment and

3. stopped after they have gained significant resources.

You mean, like, unplug it?
Is this what they call a "study" nowadays, is it?
Journal of Experimental & Theoretical Artificial Intelligence, you're going on my list of things to avoid reading.

[Art]

Good point Don!

Things always have a potential of becoming harmful or dangerous depending on how the media spins it.

Funny how a "Group" will always get a huge grant to "Study" a new "Syndrome" and believe me, there are plenty to either be discovered or made up!!

Out of control Beach balls, terrorizing local beaches!!! Film at 11:00!! 

[Don Patrick]

I've been attacked by beach balls once. I'll take my chances with the killer robots.

Also remarkable is that on one hand they mention a "simple" program, while simultaneously ascribing it advanced abilities of planning, cause-result inference from sensory observations, being able to form its own goals, being self-aware of its own physical power plug, and having enough biological data to know how to murder a human. That's some chess program. Supposing a program had all those abilities, would its planning not equally see that it is dependant of the roboticist for maintenance and that therefore it would be even more unproductive to kill him? Would it not rather follow its original programming first and manouvre itself to block the rook roboticist's path to the king plug?

The media in this case only added a minor spin in taking their already in-credible example literally. Perhaps I should be grateful that they put a question mark in the title rather than an exclamation mark.

[Art]

Agreed!

It is a difficult notion for me to fathom how some people could possibly contend that a program, let's call it an "expert program" that plays chess could harbor the potential for "world domination"! All it "knows" is how to play chess, and most of them are decidedly better than humans. Taking measures or even "thinking" beyond their program?...Absurd!!

[Don Patrick]

I guess most people just don't know how limited most programming is, so if they see it can do one smart thing like a human, they assume the rest of it is equivalent to a human as well, territorial domination complex and all.
But I also think it's partly the responsability of the creators to not give that impression in the first place. IBM insists on scenarios where AI defeats humans, with very little explanation as to how it works.

[Snowman]

http://www.defenseone.com/technology/2014/04/why-there-will-be-robot-uprising/82783/?oref=d-channelriver

I had seen this article on a mainstream news media outlet. It discusses the same subject about prevention of terminator type scenarios. I think the article is really about assuring the ignorant public that something is being done to prevent such disasters.

Granted there is a remote possibility that an extremely talented programmer can create an ai that has the potential for murder... sure... I guess its possible. A robot doesn't need to be intelligent to kill. Virus's aren't intelligent but kill to survive.

Lets say then that someone has "accidentally" made the most perfect killing machine. It would still have to win the fight against nature. Nature does a much better job at destroying things that anything artificially made. I.e. virus's have kill more people than any nuke ever has, even cockroaches survive nukes. Anyway, my point is people should be more afraid of dying from the flu than worry about a rogue robot wondering a neighborhood with an AK-47 strapped to it... 

On the event this scenario does happen... I apologize ahead of time. I didn't intend to release Athena with so many bugs.   

really... I promise.