I had a whole post here about Rutger Bregman making a similar point in his book Humankind, until I realized that I've caught him stealing from Invisibilia twice before and...yup, that's exactly the podcast he got it from. He even borrows the torchlight metaphor directly.
A terrorist facing a huge powerful army draws on another powerful weapon, empathy - but only for people like her. This is why Fritz called his book "The Dark Sides Of Empathy" - because there's a point at which empathy doesn't look anything like the universal ideal we had in our heads in the '60s. It starts to look more like tribalism, a way to reinforce your own point of view and keep blocking out all the others.
In my generation, we thought of empathy as the big, warm sun lighting the path to peace for us all. Now it operates like a torch. You shine it on your friends and use it to burn your enemies.
How long do you want to keep this up, this putting people outside the bounds of empathy? Like, how long do you want to do that because eventually - what? Like, what's the endgame of excluding some people from the possibility of empathy? Like, where do you end up?
He did point me to this Paul Bloom book that I've had on my reading list since. If empathy is whimsical emotional attachment, compassion is its rational level-headed cousin. It's not "I feel you", it's "I understand your circumstances". Which is exactly why I didn't use empathy at all earlier.
Personally, I stopped listening to Invisibilia a while ago after they jumped the shark somewhere around that episode.