When asked if the Effective Altruism (EA) movement has deviated from what he originally intended for it to look like, Peter Singer told Oxford Political Review: ‘I do think that the EA movement has moved too far and, arguably, there is now too much resources going into rather speculative long-termism.’ [1]
‘I think if we continue to focus on long term things and concern ourselves with the possibility of AI taking over in a malevolent way, it’s gonna be a very narrow movement,’ [2] he added.
Prof. Singer has been repeatedly voiced such concerns: ‘If some people want to donate to support that work [AI Safety] – excellent. But, I don’t think it’s worth investing on a large scale… I think there are better things with a better payoff today’ [3]and ‘to me it’s [AI Safety] is nowhere near as immediate as the questions of climate change – which we need to do something about now – global poverty and population growth.’ [4]
Preemptively, the piece is going to deal with some of the arguments in defence of Peter Singer’s position. Including, arguably, the main one voiced by the professor himself: ‘It’s difficult to decide between this small probability of us becoming extinct and trying to do something about that and a high probability that we can help concrete people.’ [6]
Outperforming humans
There is overwhelming evidence that the existential risks of AI are no longer ‘relatively small’. Nick Bostrom, a Swedish philosopher of AI, surveyed the top 100 most cited AI researchers on the chances of artificial general intelligence (AGI) – a machine that is capable of understanding or learning any intellectual task a human being can – being developed by 2045: the median estimate was for a one in two chance (rising to a nine in ten chance by 2075). Moreover, they estimated the chance isabout one in three (31% probability) that this development turns out to be ‘bad’ or ‘extremely bad’ for humanity [7].
Another paper, which surveyed 352 AI researchers, showed that experts believe there is a 50% chance of AI outperforming humans in all tasks by 2061 (they give a 10% chance of it happening within 7 years) [8]. And some of the most authoritative people in the field echo that the majority of AI researchers think that ‘we are likely to have a general-purpose AI around the middle of the century.’ [9]
Concerning the issue, Singer said: ‘I don’t know… I accept my lack of real knowledge on this, I’m just reporting what other people have said… From my reading, we’re still quite a long way off of the prospect of AI actually being smart enough to take us over. Somewhere in that 50 to 100 range, perhaps, which still gives us time to think about that issue…’ [10]
It is peculiar that Singer mentions the time window as a reassurance that we need not be concerned with AI. After all, he has claimed that temporal distance in and of itself should not have any moral weight. The most probable explanation is that Singer believes that the longer the time window, the higher the likelihood that humans can deal with the existential threat safely. Yet, this assumption seems to break down by its own logic: if we do not feel pressed to deal with the threat now, on account of the fact that we will have more time later, we will eventually run out of time.
A broader problem is that we are more likely to create an unsafe AI rather than a safe one, because making a superhuman-level AI that is safe involves some additional challenges on top of the challenge of creating a general-purpose AI in the first place [11]. Once machines are capable of designing other machines like them, it will result in an explosion of intelligence that will push us past the point of no return, writes Nick Bostrom [12].
So, the people in the field are unclear about what the future holds, but nevertheless, these same people think that the catastrophic risks are possible and that it is an important problem. Perhaps, then, AI Safety reseaAnd ddrch is not that ‘speculative’ after all.
Is it worth trying to reduce the AI risk?
If we ‘reduce existential risk by mere one-millionth of one percentage point, it will be worth more than 100 times the value of saving a million human lives.’ [13] The expected value of any other good actions – like helping people here and now – will be trivial compared to even the slightest reduction in existential risk [14]. This rule, known as the ‘maxipok’ rule, should have particular force for consequentialists like Singer.
One push back against this conclusion could be that we should not be concerned with events or incidents with a probability below a particular threshold (i.e., existential risks that are very unlikely to happen should be disregarded). This is best expressed in Singer’s own words: ‘The speculation that we will develop AI to such a point that it will become smarter than us and will, maybe, destroy us: firstly, it’s hard to know how likely this is; secondly, it’s hard to know how we – with our present state of knowledge – could prevent that… So, I think, we don’t know enough [about the risk of AI] to divert any funds from the existing charities.’ [15]
There are problems with each of these statements. Regarding the first point, Singer seems to be asking ‘how likely must an existential risk be for us to start taking measures to reduce it?’ According to Singer himself: ‘If the [existential] risk were 1%, that would definitely be worth doing.’ [16] It is worth mentioning that there is likely to be an implicit assumption that the probability of existential risk is estimated by competent people in a relevant field. In the case of AI, some of the world’s leading experts in the field assign 18% probability that the development of AI turns out to be ‘extremely bad — existential catastrophe.’ [17] Thus, following Singer’s own line of reasoning, the existential risk of AI passes the risk threshold.
What about the refutation that ‘we don’t know enough’ or that the future is too uncertain? Uncertainty is not a problem. The EA movement has always been about working with probabilities, which is a way of dealing with uncertainty. And AI scientists have shown that while the future is uncertain, there is a high likelihood of catastrophic risk in that uncertainty.
To pivot from my argument for a moment, there could be a couple of additional objections to the conclusion I have just drawn – again, best expressed in Prof. Singer’s own words: ‘We should not take these estimates too seriously. The overall response rate was only 31%, and researchers working in AI have an incentive to boost the importance of their field by trumpeting its potential to produce momentous results.’ [18] Indeed, the concern is reasonable: researcher bias is a common problem. Yet even if we employ the most generous correction to the current probability of 18%, we would still have a risk percentage higher than the 1% threshold Singer employs.
Let us go back to Singer’s second larger point – that ‘it’s hard to know how we could prevent’ the undesirable outcome of an AGI. This argument seems paradoxical. The more we think about how to reduce the AI risks (e.g., value alignment, reward hacking, etc.), the more we find solutions to these problems (e.g., inverse reinforcement learning, generative adversarial networks, etc.). But these solutions are possible precisely because of AI Safety research. Hence to claim that: ‘We do not need to support AI Safety precisely for the reason that we do not know how to prevent it’ is putting the cart before the horse. The fact that we do not know how to prevent an AGI currently should be a reason in favor of AGI safety research not a reason against it.
Prof. Singer noted that: ‘Negligence […] is culpable in judging the agent, how careful he was to find out what the likely consequence of his actions were.’ [20] In the case of AI – following this line of reasoning – by not paying due attention to the Safety research we all could be ‘the negligent agents.’ Are we culpable in the event of a catastrophe?
The inevitability of AGI
It seems likely that we are. According to experts, superhuman-level AI is inevitable [21]. Three assumptions support the conclusion they have reached:
The first is a premise that information processing is the basis of intelligence. It seems clearly to be the case given that we have already built narrow intelligence into our machines: its strength could be weak and limited, but at this stage all we need to do is to accept that narrow AI systems – like IBM Watson, or AlphaGo – do, indeed, demonstrate some level of genuine intelligence.
The second assumption leading to the inevitability of AGI is that we do not stand on a peak of intelligence. It is likely that the spectrum of intelligence extends much further than we currently conceive possible because many AI systems are already at superhuman-level of intelligence in their narrow tasks – it is sobering to think of: arithmetic; driving or chess (for example) humans will never be better than AI at these tasks. The challenge we face now is developing ‘flexibility’ of AI between tasks (or its generality), but not the creation of superhuman AI as such.
Third, we will continue to improve our AI systems. Certainly potential benefits of creating an AGI are huge: a more intelligent agent than we are may help us to solve (or drastically reduce) all the problems facing humanity today. In fact, the word ‘intelligence’ literally means: the ability to manipulate one’s environment to satisfy one’s objectives [22]. And, it seems, it is at the core of anything that we value – provided that we have problems we want to solve: from cancer to climate change; as long as there is a huge governmental and commercial interest; and given that the companies and governments (developing an AGI) are likely to be in a race against each other – it seems that we will not stop improving the technology. There is an argument to be made that it is probably impossible to put an end to AI research anyway: ‘As a practical matter AI research proceeds by people writing stuff on whiteboards and it’s very hard to pass legislation banning equations being written on the boards,’ explains Stuart Russell [23].
Ultimately, if intelligence is some form of information processing and if we get the appropriate algorithm right, it is likely that we develop a superhuman AGI. An argument of David Deutsch (not considering his views on AI) is relevant in that regard: anything that is compatible with the laws of nature is achievable given the requisite knowledge (i.e., the ‘momentous dichotomy’) [34].
AI risks and climate change
So, the majority of AI experts think that an AGI is inevitable [25]. Should we ignore catastrophic risks simply because most experts think they are more than 30 years away? And if so – by this logic – should we also ignore the risks of climate change? What are P. Singer’s views on the two things?
‘Probably my biggest fear today is climate change, that we are not reducing our greenhouse gasses sufficiently quickly to avoid grave risk of catastrophic changes,’ said the Professor on the first matter [26].
And talking about AI, he said: ‘To me it’s [existential risk of AI] nowhere near as immediate as the questions of climate change which we need to do something about now… From my reading we still quite a long way-off of the prospect of AI being smart enough to take us over… Somewhere in that 50 to 100 range, perhaps, which still gives us time to think about that issue.’ [27]
I would argue that, in fact, our priorities to AI safety should be equal to, if not higher than, that of climate change. The extreme scenario of climate change – given by the experts in that field – is a rise in temperature by 4°C by 2075, which would cause unprecedented heat waves, droughts, and floods, with irreversible damage to our ecosystems [28]. In the same period, AI experts think that the technology of AGI can actually threaten our existence (and they have also estimated the threat in percentage points) [29]. Thus, relying purely on the views of experts in the relevant fields – both artificial intelligence and climate change respectively – it seems that the expected value of prioritizing AI Safety is at least equal to if not higher than that of climate change.
Factual information on the latter – which is presented in the piece – is drawn from an extensive article titled The Future of Humanity, written by Nick Bostrom [30]. He mentions, that the ‘most extreme scenarios’ put forward by the UN Intergovernmental Panel on Climate Change – which is, arguably, the most authoritative body in the field – predict global warming by the end of the century ranging from +1.8 to +4°C. Bostrom concludes: ‘While this prognosis might well justify a range of mitigation policies, it is important to maintain a sense of perspective when we are considering the issue from a ‘future of humanity’ point of view.’
It all boils down to the consequences
When asked what does he do with a fairly common objection to consequentialism – that, it seems, the votes are never finally and fully in – P. Singer said: ‘You have to predict as well as you possibly can the consequences and as they got further and further out and become quite uncertain you could speculate that there will be good consequences… or bad consequences.’
‘Or you just have to say that the probabilities are so uncertain that it is nothing to take account out there, so we have to go with consequences in the near future that we can predict,’ he added [32].
I certainly agree. In essence, the only ‘objection’ of mine throughout the piece was that: it is not the case with AI that probabilities of the extremely bad consequences are so uncertain that ‘it is nothing to take account out there.’ So, could Prof. Singer be wrong that ‘there is now too much resources going into rather speculative long-termism?’
I do not know. But, with due respect, some phrases of P. Singer – such as ‘malevolence’ and ‘hostility’ of AI – do raise some concerns that his view does not engage with some of the core arguments in support of the AI Safety research.
Yes, the future is yet to unfold and our prospects may turn out to be way more optimistic than predicted by the AI researchers. Yet, I would contend that the probability that large groups of experts (in the relevant field) are right is still higher than the probability that any other person independently will be right. And, unfortunately, the predictions are nothing close to optimistic. If Bostrom is right, the risk of a bad AGI is worth taking seriously, here and now.
Bibliography
[1] Peter Singer, Interview Oxford Political Review: https://www.facebook.com/589176081541261/videos/453226281889939/?t=321, at 5:22 [2] ibid., at 5:55 [3] Peter Singer on effective altruism, vegetarianism, philosophy and favourite books. Book Person #27: https://www.youtube.com/watch?v=0NQUo834df4, at 19:17 to 19:46 [4] Philosopher Peter Singer on AI, Transhumanism and Ethics: https://www.youtube.com/watch?v=tcs9p5b5jWw, at 18:22 to 18:44; 29:34 to 30:20see also, Peter Singer on Good Lives, Good Futures, and Good Philosophical Writing: https://www.youtube.com/watch?v=fCmPro1uknQ&t=2s, at 23:48 to 25:51
[5] Peter Singer on effective altruism, vegetarianism, philosophy and favourite books. Book Person #27: https://www.youtube.com/watch?v=0NQUo834df4&t=486s, at 14:20 to 14:52 [6] ibid., at 14:56 [7] Nick Bostrom & Vincent C. Müller, Future Progress in Artificial Intelligence – A Survey of Expert Opinion: https://nickbostrom.com/papers/survey.pdf, abstractsee also, When Will AI Exceed Human Performance? Evidence from AI Experts (Oxford University, Yale University, last revised May 2018): https://arxiv.org/pdf/1705.08807.pdf, p.13
see also, S. Russell and A. Dafoe, Yes, We Are Worried About the Existential Risk of AI (2016): https://www.technologyreview.com/s/602776/yes-we-are-worried-about-the-existential-risk-of-artificial-intelligence/
[8] When Will AI Exceed Human Performance? Evidence from AI Experts (Oxford University, Yale University, last revised May 2018): https://arxiv.org/pdf/1705.08807.pdf, abstract, p. 2 [9] Stuart Russell on Artificial Intelligence: What if we succeed?: https://www.youtube.com/watch?v=qXgKFRq3JRA&t=267s, at 1:17:22see also, Stuart Russell: Long-Term Future of Artificial Intelligence | Artificial Intelligence (AI) Podcast: https://www.youtube.com/watch?v=KsZI5oXBC0k, at 1:10:45
see also, What happens when our computers get smarter than we are? | Nick Bostrom: https://www.youtube.com/watch?v=MnT1xgZgkpk at 4:27
[10] Philosopher Peter Singer on AI, Transhumanism and Ethics: https://www.youtube.com/watch?v=tcs9p5b5jWw, at 56:36 to 56:58; 18:32 to 18:45 [11] What happens when our computers get smarter than we are? | Nick Bostrom: https://www.youtube.com/watch?v=MnT1xgZgkpk, at 15:15 [12] Nick Bostrom, Superintelligence: Paths, Dangers, Strategies (Oxford University Press), Chapter 8see also, Artificial intelligence: ‘We’re like children playing with a bomb’: https://www.theguardian.com/technology/2016/jun/12/nick-bostrom-artificial-intelligence-machine
[13] The end of humanity: Nick Bostrom at TEDxOxford: https://www.youtube.com/watch?v=P0Nf3TcMiHo, at 6:00 to 6:36 [14] Phil Torres, Space Colonization and Suffering Risks – Reassessing the “Maxipok Rule”: https://doi.org/10.1016/j.futures.2018.04.008, at p.3see also, The end of humanity: Nick Bostrom at TEDxOxford: https://www.youtube.com/watch?v=P0Nf3TcMiHo, at 7:34
[15] Peter Singer on effective altruism, vegetarianism, philosophy and favourite books. Book Person #27: https://www.youtube.com/watch?v=0NQUo834df4, at 18:17 [16] ibid., at 18:12 [17] Nick Bostrom & Vincent C. Müller, Future Progress in Artificial Intelligence – A Survey of Expert Opinion: https://nickbostrom.com/papers/survey.pdf, p.12[18] Peter Singer, Can artificial intelligence be ethical?: https://www.weforum.org/agenda/2016/04/can-artificial-intelligence-be-ethical
[19] When Will AI Exceed Human Performance? Evidence from AI Experts (Oxford University, Yale University, last revised May 2018): https://arxiv.org/pdf/1705.08807.pdf, p.5 [20] Making Sense with Sam Harris #48 – What Is Moral Progress? (with Peter Singer): https://www.youtube.com/watch?v=yAwcpFGu2Y4&t=3246s, at 1:10:55 to 1:11:10 [21] Nick Bostrom & Vincent C. Müller, Future Progress in Artificial Intelligence – A Survey of Expert Opinion: https://nickbostrom.com/papers/survey.pdf, abstractsee also, When Will AI Exceed Human Performance? Evidence from AI Experts (Oxford University, Yale University, last revised May 2018): https://arxiv.org/pdf/1705.08807.pdf, p.2
see also, S. Russell and A. Dafoe, Yes, We Are Worried About the Existential Risk of AI (2016): https://www.technologyreview.com/s/602776/yes-we-are-worried-about-the-existential-risk-of-artificial-intelligence/
[22] Merriam-Webster, “intelligence”: https://www.merriam-webster.com/dictionary/intelligencesee also, Stuart Russell on Artificial Intelligence: What if we succeed?: https://www.youtube.com/watch?v=qXgKFRq3JRA&t=267s, at 47:52
[23] Stuart Russell on Artificial Intelligence: What if we succeed?: https://www.youtube.com/watch?v=qXgKFRq3JRA&t=267s, at 22:20 [24] David Deutsch, The Beginning of Infinity: Explanations That Transform the World, Chapter 9: “…it is universally true that either the laws of physics forbid eliminating it in a given time with the available resources or there is a way of eliminating it in the time and with those resources.”see also, Making Sense with Sam Harris #22 — Surviving the Cosmos (with David Deutsch): https://www.youtube.com/watch?v=2dNxxmpKrfQ&t=1552s, at 28:22 to 29:00
[25] Nick Bostrom & Vincent C. Müller, Future Progress in Artificial Intelligence – A Survey of Expert Opinion: https://nickbostrom.com/papers/survey.pdf, abstractsee also, When Will AI Exceed Human Performance? Evidence from AI Experts (Oxford University, Yale University, last revised May 2018): https://arxiv.org/pdf/1705.08807.pdf, p.2
see also, S. Russell and A. Dafoe, Yes, We Are Worried About the Existential Risk of AI (2016): https://www.technologyreview.com/s/602776/yes-we-are-worried-about-the-existential-risk-of-artificial-intelligence/
[26] Philosopher Peter Singer on AI, Transhumanism and Ethics: https://www.youtube.com/watch?v=tcs9p5b5jWw, at 17:08 [27] ibid., at 18:22 to 19:01 [28] Solomon, S., Qin, D., Manning, M., and al., e. (2007) Climate Change 2007: The Physical Science Basis. Contribution of the Working Group I to the Fourth Assessment Report. Edited by Intergovernmental Panel on Climate Change (Cambridge: Cambridge University Press). [29] Nick Bostrom & Vincent C. Müller, Future Progress in Artificial Intelligence – A Survey of Expert Opinion: https://nickbostrom.com/papers/survey.pdf, abstract [30] Nick Bostrom, The Future of Humanity: https://nickbostrom.com/papers/future.htmlsee also, Artificial intelligence: ‘We’re like children playing with a bomb’: https://www.theguardian.com/technology/2016/jun/12/nick-bostrom-artificial-intelligence-machine
[31] Will MacAskill, What are the most important moral problems of our time?: https://www.youtube.com/watch?v=WyprXhvGVYk&t=181s, at 9:54 [32] Making Sense with Sam Harris #48 – What Is Moral Progress? (with Peter Singer): https://www.youtube.com/watch?v=yAwcpFGu2Y4&t=3246s, at 1:31:48 to 1:32:33