a thoughtful web.
Share good ideas and conversation.   Login or Take a Tour!
hyperflare  ·  1661 days ago  ·  link  ·    ·  parent  ·  post: Threat from Artificial Intelligence not just Hollywood fantasy

Or I just create a tribe of humans and let human civilization take its course until we are at a modern standard and gun them all down? I.e. what would happen if we had self-improving AI

I don't know about the key to destruction part. If we end up in another cold war, and someone decided to use a slaved AI to manage their defenses better than the enemy. Posit that a slaved AI is further advanced enough over humans that it could defend the country better than any human team could attack it. Then the enemy upgrades to slaved AI for their attack as well. You could either choose to unslave your AI (making it more powerful, seeing as it can improve itself now) and be safe from attack, or you could keep it slaved and hope your enemy doesn't have a bad day. I think that if there's a big enough benefit to doing something like that, people will do it. Practically, we all handed that power off to someone already (like the president). I don't trust the president of the US more than I do an AI, except insofar as he has to preserve his own life.