I think liking the music is fine, but depending on how much you identify with the lyrics, you may be a misogynist (not you specifically). The lyrics are not only creepy, they are self absorbed and allude to emotional abusiveness (as in the mentioned No Other One). It's a damn catchy album, but the lyrics seriously make me cringe.
If lyrics about "disgusting" subjects make you cringe, it seems you'll have trouble with a fair amount of the western literary canon. No Other One is a pretty good song, sound-wise; its message is stupid but certainly not fantastic (in the original sense of the word). Some people think like that, so crucifying Cuomo for writing a creepy song in an era of creepy songs makes me wonder how much music the authoress has actually listened to. Does she have issues with Nirvana? Pavement? A thousand other bands in the grunge and neo-grunge genres? It's just a song on an album. It sounds good. Some people's writings are self-absorbed -- nothing wrong with that. Identifying with a (potential) pedophile in Across the Sea is weird, yeah, but how many people actually do identify with it?
It's not just a song, it's basically the whole album, and frankly Nirvana and Pavement don't even come close to the overt creepiness/abusive vibe of Pinkerton. You can sweep it all under the rug if you want, I guess.