I used to be convinced by this argument. I am not convinced by it anymore. I've spoilered my argument in case you don't feel like having to deal with an existential crisis.

When you consider how your enjoyment of good events may be causing bad events for other people, such that the "net enjoyment" you put into the world is negative, can it really be said that you are living an ethical life? I used to be unable to experience empathy, and I still have a very difficult time with empathy as a concept and as a subjective experience. But when empathy is a factor, other people's happiness becomes a part of my own happiness. Their distress also becomes my distress. If my actions send ripples throughout the world, then am I not responsible for those ripples? For those second-order and third-order effects of my actions?

I'll give an extreme example: rape jokes. Rape jokes can get a laugh from a certain crowd. If you're part of that crowd, then telling a joke about rape and having it be well-received by your peers will be a good event. But then the rapists in that crowd will take it as a signal that rape is normal, that it's accepted by you and your peers. And so the chance that they will hurt someone is increased. The cost of your rape joke is to increase the chance that a woman will be raped.

"Social responsibility" means being socially aware enough to understand things like the rape joke example, to understand how our actions affect other people and have higher-order consequences which shape our culture, and then using that understanding to inform your morality. Rape jokes are disgusting. You should be disgusted when you hear them. You should judge people who don't think they're a bad thing.

Rape is an extreme example but this applies to anything that could make someone else's life worse just so you can have some small benefit.