Here's the problem:

When you put your foot down and say "I won't engage in this immoral claptrap" you've automatically added yourself to the outgroup.

Unless you are in a position of leadership, or have enough charisma to assume the mantle of leadership, or are a compelling enough orator that you can make everyone cease and desist in the behavior without resenting you in such a way that they never again want to gossip, all you've done is made yourself the enemy. Period. Full stop. The gossip shall continue. But now it's about you.

Unhealthy work environments? I haz them. I had a boss single me out for drug testing because they realized too late that I wasn't a born-again Christian like everyone else they hired. I worked at a place where one of my coworkers had to go to HR because he was sick of being shot in the face with airsoft pistols (it didn't change anything). Same job? I was the only person that hadn't seen Melissa's tits multiple times because when we went to trade shows I walked the floor like a citizen, gathered information like a citizen, and then went back to my hotel to rest; I didn't spend all day at bars and strip clubs.

And when it came time for layoffs, the guy who hadn't seen Melissa's tits was the first to go, even though it cost the company $20m in billable clients.

You can't fight the tide. If you aren't management, stop. You're participating in a work environment, not policing it and when you try, you simply reveal yourself as a narc.