a thoughtful web.
Share good ideas and conversation.   Login or Take a Tour!
kleinbl00  ·  1790 days ago  ·  link  ·    ·  parent  ·  post: Threat from Artificial Intelligence not just Hollywood fantasy

    Uh, what did you expect? A step-by-step guide to world domination by the hyperintelligent AI I keep in my calculator?

That's exactly what I expected. That's what I asked for. That was the bounds of the discussion, the ground rules of the thought experiment. IF: hyperintelligent AI AND: world domination is the goal THEN: how, exactly, would it be accomplished?

And your response, as well as your responses above, all say "I don't know but I'm sure it would happen."

And that's my beef. It won't. Waving hands and insisting it to be so is not the same as an actual, practical methodology towards nefarious machine behavior. Here, look:

    I actually doubt that I'd be able to infect a computer in a nuclear plant with something as primitive as a USB stick, but on the off chance that it did work, I'd be easy to fake read-outs to the controllers (via the SCADA). Alternatively, you could just uncouple the turbines and watch them blow. Suddenly your power has nowhere to go and you need to initiate emergency shutdown. Even if I can't do anything, I can decieve the controllers. If I do this to all plants, it will succeed in at least a few.

Have you ever been in a powerplant? Or a large factory? Or a pump station? Or a refinery? Or any large facility of the type typically suggested for targeting? They're full of giant mechanical shut-off valves and giant hand actuated breakers and giant full manual safety interlocks because there's no advantage to automation on the scale you need to cause turmoil. The attack on Natanz worked because the centrifuges there were specific devices with a specific job that can't be done without computer control. They were hacked, and they failed as a result. The plant didn't blow up, the grid didn't go dark, the T-1000 didn't stalk Tehran.

I can tell that you think you understand the fundamentals here, but you don't. "It'd be easy to fake read-outs to the controllers (via the SCADA)" is absolutely true... but if you want to do real damage, you need to co-opt this guy:

Repeat for every point you've made, frankly. "All the things, at once" compounds the problem, not making it easier. The world is not autonomous. The world is highly instrumented. Not the same thing. It's easy to think it's the same thing, because Hollywood is big on that idea.

But it's Hollywood.

The "T-1000 with a shotgun" is not a natural conclusion. It's a flight of fancy. The fear of malevolent AI is rooted in a fundamental misunderstanding of how many people actually keep your world running.

Click this link. Count the cars. That's how many people it takes to keep the poop flowing in West Seattle.

Now click this link. Count the cars. That's how many people it takes to turn oil into gas for about a million and a half people.

Now click this link. Count the cars. That's how many people it takes to keep a 500MW coal-fired powerplant running.

These aren't jobs programs. This isn't welfare. These are trained professionals keeping your toilet flushing, keeping your lights on, and keeping gas in your tank.

Keeping your malevolent AI from picking up the shotgun.