Thinking Inside the Box: Controlling and Using an Oracle AI

被引：39

作者：

Armstrong, Stuart ^{[1
]}

Sandberg, Anders ^{[1
]}

Bostrom, Nick ^{[1
]}

机构：

[1] Univ Oxford, Fac Philosophy, Future Humanity Inst, Oxford OX1 1PT, England

来源：

MINDS AND MACHINES | 2012年 / 22卷 / 04期

关键词：

Artificial intelligence; Superintelligence; Security; Risks; Motivational control; Capability control;

D O I：

10.1007/s11023-012-9282-2

中图分类号：

TP18 [人工智能理论];

学科分类号：

081104 ; 0812 ; 0835 ; 1405 ;

摘要：

There is no strong reason to believe that human-level intelligence represents an upper limit of the capacity of artificial intelligence, should it be realized. This poses serious safety issues, since a superintelligent system would have great power to direct the future according to its possibly flawed motivation system. Solving this issue in general has proven to be considerably harder than expected. This paper looks at one particular approach, Oracle AI. An Oracle AI is an AI that does not act in the world except by answering questions. Even this narrow approach presents considerable challenges. In this paper, we analyse and critique various methods of controlling the AI. In general an Oracle AI might be safer than unrestricted AI, but still remains potentially dangerous.

引用

页码：299 / 324

页数：26

共 38 条

[1] Anderson M., 2011, Machine Ethics
[2] [Anonymous], 1982, JUDGMENT UNCERTAINTY
[3] [Anonymous], 2005, SINGULARITY IS NEAR
[4] Armstrong S., 2010, TECHNICAL REPORT
[5] Asimov I., 1950, I, Robot (The Isaac Asimov Collection)
[6] Are we living in a computer simulation?
Bostrom, N
[J]. PHILOSOPHICAL QUARTERLY, 2003, 53 (211) : 243 - 255
[7] Bostrom N., 2012, MINDS MACHI IN PRESS
[8] Bostrom N., 2011, REV CONT PHILOS, V10, P44
[9] Bostrom N., 2012, SUPERINTELLIGE UNPUB
[10] Bostrom N., 2000, C MANILANA PDCIS, V7

← 1 2 3 4 →