An Open Letter to Nick Bostrom

Nick:

Let me start by complimenting you on a bravura performance.  I’ve read dozens of books and articles on ASI but Superintelligence is clearly in a category of its own.  I have no doubt that your work will stand as the definitive reference in the field for some time to come.

On the other hand, I do have a significant objection; if only to the work as a whole.  You make a compelling case for working together to mitigate the unprecedented peril of ASI—a sorely needed case, truth be told!—but I can’t help but think that, even so, your book is suffused with too much optimism.  To resort to bad haiku:

Fearsome ASIs
Can undoubtedly be tamed
We will make them safe!

At the risk of being a Cassandra, I fear that such optimism is entirely wrong!

Continue reading

The Singularity Starts Now

“Everyone talks about the Singularity; but no one does a thing about it!”

Nothing!  Not a laugh, not even a snigger.  You’d think that a theatre full of geeks would contain at least a single person who’d thought that I was funny; but no.  They all just sat there, like veal; probably pissed that an unknown speaker was delaying Kurzweil’s entrance.  Under any other set of circumstances, I would have been a nervous wreck; convinced that I’d lost them.  I had a trick or two up my sleeve, though, so I pressed on.

Continue reading