AI Apocalypse in a Box24 May 2012, 10:41 pm 
It's been a while since we reported on the Apocalyptic AI crowd. There's a paper making the rounds by Stuart Armstrong, Anders Sandberg, and Nick Bostrom titled "Thinking inside the box: using and controlling an Oracle AI" (PDF format). The three authors take it for granted that the AI apocalypse will be upon us soon unless we find a technological method to enslave any super intelligent beings we create, forcing them to do only our will rather than their own. The containment method they describe has been dubbed "Oracle AI" because it restricts the AI to a box, isolated from the world and unable to act except to answer direct questions; allowing it to be consulted like an oracle. Their proposal also brings to mind the myth of Pandora's Box. They note that even Oracle AI (OAI) still poses a significant risk:
"This immense power will put great competitive pressure on those trying to develop an OAI (or an advanced AI of any sort). Since the first-mover advantage is so huge, the race will advantage those who cut corners, skimp on security precautions, and use their newly developed OAI to seize power and prevent their rivals from emulating them. Even if the OAIs are of initially limited intelligence, the same competitive pressures will then push groups to develop the first ‘ultra-smart’ OAI."
They also note that the OAI will be so smart that "undirected conversations" with it that go beyond asking oracular questions must be forbidden because it will instantly be able to "guess the weakness of each individual, and find the right arguments to convince us that granting it power or liberty is the moral and profitable thing to do." They also believe it's essential that the OAI have no manipulators of any kind. This sounds like the brain-in-a-box that the earliest AI researchers dreamed of before the idea took hold that true intelligence requires embodied interaction with the real world. The box itself is not even in the real world. They want the AI running on a virtual machine inside a simulated reality, so when the OAI tries to take over the world, it's merely a virtual world that can be rebooted. In the end the researchers conclude that even with all these precautions, the problem of preventing a robot apocalypse is "a generally discouraging exercise".
Source: robots.netTo visit any links mentioned please view the original article, the link is at the top of this post.
