You're definitely not the only one to care about what people think of them.
There's a whole bazillion dollar industry dedicated to making sure that everyone worries about what people think of them (media). And another gazillion dollar industry that does quite well with the fallout (pharma).
Don't go worrying that you're engaging with society in the only way society has trained you to. It's sure as fuck not your fault.