Primary supervisor
David DoweDiscussion of dangers of artificial intelligence (or artificial superintelligence, ASI) being an ambitious subordinate and ultimately taking over control from humans goes back at least as far as R J Solomonoff (1967). R J Solomonoff (1985) gives an approximate time-frame in which this might occur.
D L Dowe and A R Hajek (1997a-b, 1998) discusses understanding and learning in terms of compression and the Bayesian information-theoretic minimum message length (MML) principle, with similar ideas in the independent and contemporary J Hernandez-Orallo and N Minaya-Collado (1998). S Legg and M Hutter (2007) later proposed a universal measure of intelligence via algorithmic information theory (AIT) or (Solomonoff-)Kolmogorov complexity and weighted average of performance across varied environments. J Hernandez-Orallo and D L Dowe (AI Journal, 2010) refined this to give a test which has been stated to be universal and which could be stopped any time to give an anytime estimate of intelligence. Later joint work by N Chmait, D L Dowe, D G Green and Y-F Li would quantify the intelligence of collectives, and show cases where a collective of less intelligent agents can outperform a single more intelligent agent.
Humans try to improve their intelligence - or collective intelligence of extended community - by use of machine intelligence. There is a balance between getting machines to be more intelligent, go faster and doing things in ways which humans might understand (on the one hand) and (on the other hand) being able to verify and control what the electronic machine intelligence is doing.
Some believe that humans will be able maintain control. Others might feel that humans have effectively already lost control and that what ensues will not be good for humans.
Turing Award winner and Nobel Laureate Geoffrey Hinton is often quoted as having said: "If you want to know what life's like when you're not the apex intelligence, ask a chicken."
The scenario described here by Hinton is also discussed in R J Solomonoff (1967, 1985), D L Dowe (2013), S W Hawking (2014), D L Dowe (2014) and many others.
Aim/outline
We will consider simulation of a variety of scenarios of (pseudo-)humans and computers.
There will be a variety of degrees of co-operation between the (pseudo-)humans in the different simulations.
The general questions we address are (at least initially)
(i) how far humans can extend their intelligence while still being sure of having control,
(ii) how far humans can extend their intelligence while (quantitatively) still being quite sure of having control.
We hope to submit (and publish) work to a reputable publication outlet in (e.g.) AI safety.
URLs/references
N Chmait, D L Dowe, D G Green and Y-F Li
D L Dowe (2013)
D L Dowe (The conversation, 2014)
D L Dowe and A R Hajek (1997a)
D L Dowe and A R Hajek (1997b)
D L Dowe and A R Hajek (1998)
J Hernandez-Orallo and D L Dowe (AI Journal, 2010)
S W Hawking (2014)
J Hernandez-Orallo and N Minaya-Collado (1998)
S Legg and M Hutter (2007)
R J Solomonoff (1967)
R J Solomonoff (1985)
Required knowledge
Ability to program and perform simulations.
Good marks in subjects involving mathematics at least to 1st year undergraduate level (or exceptional mathematics marks at secondary school level).
Understanding of - or strong desire to develop understanding of - information theory and environment complexity.
Understanding of - or strong desire to develop understanding of - information theory and task complexity.