★★★★★ 5
A well-written but perhaps too late warning
Format: Kindle
I wavered between 4 stars or 5 but ended up with 5 despite some reservations. The author has put a great deal of work into this book, which includes interviews with and intriguing anecdotes about most of the leading figures in the AI revolution. I did not know, for example, that the term “singularity” was coined as an analogy to the event horizon of a black hole – the point beyond which we cannot see the future. This is not the deepest or most technical book on this topic: that award goes to Nick Bostrom’s Superintelligence. It also ignores the short to medium term issue posed by even sub-human AI -- the millions of job losses (hundreds of millions globally) likely to occur in the next 10 to 20 years. It focuses instead on the risks of super-intelligent AI, AI that exceeds – soon by orders of magnitude – human level intelligence. It is nevertheless a superb book for its intended purpose: raising public awareness of the existential risk posed by this development. AI, the author says, is the cuckoo chick in the nest. The AI community built the nest and is now busily feeding this strange chick. Mesmerized by its open mouth, they ignore the mortal danger it poses to their own progeny. Even when they know what will happen in the end, they cannot quite believe it. Only intervention by the non-technical public has any chance at all of short circuiting this process.
Against these many good points, I would have liked to hear the author’s take on what I think is the critical question overlooked both by Kurzweilian optimists and AI skeptics. Both the notion that we will somehow “merge” with AI and the notion that AI will eat us alive depend on the assumption that silicon-based intelligence can have conscious awareness. We certainly wouldn’t want to merge with anything that would result in our becoming permanently unconscious, and Barrat repeatedly assumes that AI will be “self-aware,” a state that first requires being “aware,” that is phenomenally conscious. The unasked question is whether AI, as it is currently being developed, can have that capacity. IBM’s Watson may be good at Jeopardy but there is no reason to believe that it knows it is good at Jeopardy, or feels good at being good at it. By contrast, honey bees appear to become depressed when they are shaken. This suggests that there is something fundamentally wrong about the notion that current AI, as it becomes more intelligent, will “automatically” become conscious. The best current theory of consciousness – integrated intelligence theory – suggests that a computer can become conscious but only if it is wired very differently from the ones we currently have.
Nevertheless, this is still an excellent book, so in the end I thought the 5 star rating was deserved.
WAS THIS REVIEW HELPFUL?YesReportShare
Reviewed in the United States on June 14, 2015
