... sigh ... So, their little baby AI, as it learns to be more human, is going to be child-like and "feeble-minded" (Bostrom's words) for its entire "life" ... and therefore should be exterminated, as he says human beings should be. Or, lemme guess, your little baby AI, Mr Bostrom, is MORE precious than an actual living human being, and therefore should be given the resources to thrive, rather than the human being. It's like these "long-term-ists" can only process any thought through three steps, and then they just wave their hands and yell 'PROFIT!' at the end. They'd be so easy to beat at chess... "...In 2011, Moskovitz was among the early-funders of AI and robotics firm, Vicarious, which is trying to create machine learning software that can think and learn like a human, based on the computational principles of the human brain..."
I know it's just a side-remark, but human chess is much less about looking far than knowing where to look. It's really a hump curve for most players: beginners work at depth 2-3, intermediates push it to 10-16, experts do 3-5 without wasting a second on what doesn't seem promising.They'd be so easy to beat at chess...
What I've never understood is why? Why do we want AI that thinks and acts like human intelligence (setting aside the questions of what makes human intelligence human)? Humans are amazing at many things, but are deficient is many areas. Writing novels and making amazing creations in all sorts of media are one of the areas we are decidedly NOT deficient in. All this human modeling from the Blue Brain onward has always struck me as such a navel-gazing waste of resources.
Because the thinking AI/red heifer will demonstrate/fulfill predictions/prophecy about the coming Singularity/Rapture which will sweep all righteous/righteous into an exciting new future/paradise in which the skeptics/nonbelievers will face extinction/tribulation prior to establishing our neoutopia/Kingdom of Heaven.
The altruistic long-future perspective says that the fewer humans have to work in menial jobs, the more free they will be to use their big brains for other things; art, science, whatever. There are all kinds of shit jobs that can be replaced with a moderately capable AI, and those people can now go find meaningful work elsewhere. Of course, if you are an American there is no social safety net to allow you time to develop your art/idea/passion/invention, so you need to come from a rich family who will underwrite you until your product hits the market. But hey - as long as daddy is rich and you have all the free time in the world, you can employ AI to do all the "meaningless" tasks you don't want to do. Doesn't that sound like a wonderful utopia? Yeah nah... me neither.
Effective Altruism argues "I don't need to think about the person in front of me, I need to think about all the hypothetical millions that will come after him (that will be of my stock, educated by me, of my tribe, in my clan, who follow my beliefs)." that's the whole schtick. It's yet another selfish-filter, like libertarianism or anarchism. Apropos of nothing this? Is only as hilariously awesome as it is because it is so blissfully lacking in self-awareness. It is a Spinal Tap-grade examination of the fundamental principles of "fuck around and find out." “This is the true story, of seven selfish assholes, deciding to move to Mexico, grift off each other, and have their lives taped. Find out what happens, when people stop being polite, and start getting Libertarian… The Libertarian World!”