First, human software engineers could sit down and design an AI to fulfil a certain function, and then it runs amok in some unforeseen way. An extreme scenario that sometimes gets mentioned is that of a super-AI being asked to solve a problem and then stupidly marshalling any and all resources on Earth to produce the solution, in the process inadvertently killing off all humans.
The advantage of this scenario is that programmers are constantly doing precisely that, that is designing specialised AI to solve individual problems. The downside is that programmers are constantly doing precisely that, that is designing specialised AI to solve individual problems, and so far nothing untoward has happened. And by extension there is no reason whatsoever to assume that it ever will.
That is because the AI in question is specialised to do one thing and thus generally unable to do anything else, because the programmers know very well how to give it the right set of instructions, and because the AI does not actually have more than the required resources at its disposal. A chess computer is potentially much better than the best human at playing chess but that does not mean it will suddenly be able to take over the USAmerican nuclear arsenal. An autopilot is potentially much better than the best human pilot but that does not mean it will have either the motivation or the capability of electrocuting random humans. Etc.
The idea that once the chess computer exceeds a certain threshold of chess playing capability it could suddenly reach out and decide to do terrible things is just magical thinking, another example of treating computing power as magic pixie dust.
The second option for the generation of AI, and one that is a very popular subject of speculation amongst those singularitarians who are unhealthily obsessed with immortality, would be to emulate a human brain. This is where we really enter the realm in which we have to hand-wave objections with regard to the plausibility of it all, but on the other side it has the advantage that it is immediately clear what would motivate humans to do it, because obviously we have an incentive to study the human brain.
So let's just assume in a few decades the brain would be so well understood and computers so advanced that one could simulate a human brain; or alternatively, that one would somehow copy the workings of the brain in some hardware fashion, perhaps as a building-sized signalling network. What would that give us?
Well, I cannot help but notice that it would NOT give us a super-human AI but only a human one. Also, it would NOT give us a self-improving AI. So really all the ingredients of MIRI's doomsday scenario are missing. There are more problems but well, moving on.
Finally, the third scenario that I can assign some plausibility to - again, hand-waving the question whether sentience can be achieved outside of biological structures - is that computer scientists would somehow evolve a super-intelligence with independent motivations. In contrast to the first scenario, it might produce something very flexible and unforeseen, and in contrast to the second scenario, it might produce something really alien and potentially superhuman.
This is the best starting point for dangerous and hostile AI, but several issues remain. Perhaps the most obvious one is: Why the hell would anybody do this? If you wanted to produce AI to do something for us it would be much more reasonable to use the design approach above, so the only reason could be academic curiosity. But that brings us to the second objection: In such a scenario, the AI would be sitting somewhere in a computer and interact with the scientists who built it, and they would treat it a bit like a lab rat.
This is where LessWrongians and suchlike would mention the danger of some stupid scientist "releasing" the AI from the lab into the world where it could take over our hardware and wreak havoc. But then we are once again entering magic pixie dust territory. It is simply assumed that "a lot of intelligence" means that the AI can (1) behave like a small, specialised computer virus (2) while still maintaining coherent conscious control over all its operations, that (3) it would be so good that nobody could stop it, and that (4) the technology it could reach in that way could actually cause significant damage. Those are a lot of assumptions.
Intelligence is not actually some all-purpose superpower. Imagine the most intelligent human being on the planet, and then imagine her even more intelligent. Now tell me: can she fly an aeroplane? Well, it depends, doesn't it? If she learned how to do it, she can; if not, she can't. Just being intelligent doesn't mean much except she might be able to qualify for the flight exam after less hours. If she has the right kind of talents that is; maybe she is more of a language genius.
And what would an AI actually do against us? Have the screen of my PC snarl at me? If it launched nuclear weapons the resulting blasts will take all electronics out, and that was that. Also, shutting down power would solve the problem right away.
But that is already granting too much. It isn't even plausible that an evolved AI would acquire the necessary capabilities and the motivation to turn on us humans because that is not how evolution works. The difficulty of understanding how evolution works is of course a perennial problem. Many people assume that things always evolve to be generically "better", and in a way they do, but it is important to realise that that can mean very different things.
Take domesticated animals and plants, for example. Protected and nourished by humans, they have quickly evolved characteristics that increase the likelihood of being retained in human cultivation. Animals get fatter and tamer, plants lose the ability to disperse their seeds, certain breeds of dogs look cuter, and so on. And most of these adaptations are pretty counter-productive outside of human cultivation or are traded off against things that are. That is because selection is only for being useful to humans and for being retained in human cultivation, not, for example, for intelligence, strength, or speed.
Likewise, an evolved super-AI would be good at whatever the selection pressures were about. Maybe it will have been evolved to be good at mimicking a human, but then it will be guaranteed to not be good at invading the internet, simply because that trait was not selected for. So essentially the only way of evolving a super-AI capable of committing genocide against humans is to actively select for it, and even that would require an unrealistic effort. In effect one would have to simulate the entire planet down to every crook and nanny a few hundred or thousand times in parallel (for sufficiently large AI population size).
Ultimately, I cannot shake the feeling that the singularitarian scenario for the generation of hostile super-AI is a fourth one, which works something like this:
- Humans build more powerful AI,
- MAGIC HAPPENS,
- The AI has spontaneously developed its own motivations and can do things that are physically impossible.
No comments:
Post a Comment