People frequently have this idea of rights as a zero sum game. "If I give them rights, I'll lose some of mine!!" Of course it's not true.
Worse, though, is this idea of 'straw-feminists': bra-burning, second-wave, man-hating 'oversensitive' types that want men to apologize for being men. These people don't really exist, and if they do, they're a minority, and their ideas are not part of what feminism is. The internet loves to spread fake stories about them in the same way they spread fake stories about black girls named "La-a" and shit. It's a discussion of philosophy and things like 'patriarchy' and 'rape culture' are concepts of subconscious, systemic issues; no one believes there's a conspiracy of men talking and keeping women down, that's ridiculous, but people can't see things in a long-view or a non-literalist way. It's the same thing when we talk about problems with the institutional form of 'men', and they see it as 'all men' and take offense (and then, of course, that's the issue of making a problem about violence and oppression about how they feel when you say those things, because all attention has to be on men at all times.)
Feminism is easy: do you believe in equality? Feminist. Better: Do you want to be a good, not 'white, privileged feminist'? Do you believe in all equality, for women of color as well? Intersectional feminism. It's fucking important.