Nick:

Let me start by complimenting you on a bravura performance.  I’ve read dozens of books and articles on ASI but Superintelligence is clearly in a category of its own.  I have no doubt that your work will stand as the definitive reference in the field for some time to come.

On the other hand, I do have a significant objection; if only to the work as a whole.  You make a compelling case for working together to mitigate the unprecedented peril of ASI—a sorely needed case, truth be told!—but I can’t help but think that, even so, your book is suffused with too much optimism.  To resort to bad haiku:

Fearsome ASIs
Can undoubtedly be tamed
We will make them safe!

At the risk of being a Cassandra, I fear that such optimism is entirely wrong!

Continue reading →