Personally I’m still amazed that these supposedly luminary AI doomers can all believe we need to prevent AI from going rogue in a way that will end humanity, without actually proving that’s even possible in the first place.

Here's the fundamental dodge at the heart of "rationalism", "effective altruism" and the whole TESCREAL constellation of sociopathic bullshit:

IF: I can envision a future in which my current actions, no matter how abhorrent, will be viewed as necessary or noble

THEN: I'm not required to give the first fuck about anything happening right in front of me.

I can burn this young lady alive because obviously she's a witch. I can ignore poverty in the world and focus on my own enrichment because I am spreading the gospel of the one true God who will save their souls. I can plot the murder of anyone opposed to my thinking because in the distant future everyone will acknowledge how crucial I was towards advancing the goals of humanity as it is then understood.

Pretty sure that's the reason for all the big words and grandiose concepts - it allows you to disconnect your actions and beliefs from the conventional societal markers.

Liches have trouble thinking clearly about paths through probability space that conflict with their phylactery, and the more conjunctive a mission it is to make true their phylactery, the more bits of epistemics will be corrupted by their refusal to look into that abyss.

God's anointed are troubled by moral concerns in the mortal world, but the more they ignore these concerns instead of rejecting them out of hand the more they lose God's voice.

It's all vengeful zealot bullshit, the TESCREAL dipshits have just wrapped themselves in a cloak of pseudomath.