Yes and no.

Both men and women's bodies fall on a spectrum. Some girls are curvy, some aren't. Some guys are curvy too.

Women don't need to be in a curve fitting top to look nice. The top I am in right now is "peasant style" and not fitted. I see "boyfriend style" tops in clothes stores all the time, they're often literally men's stuff that have had minor cosmetic alterations (like the rolled up arms stitched in place). One of the current fashions these days is to get an oversized top and wear it with leggings. And not to forget, I've helped organize events where tshirts were a thing. Many many women attendees opt for the non-babydoll styles, and smaller built guys even opt for babydoll instead of unisex.

Pants are a slightly different thing. Holy fuck is it hard to find a pair of jeans that fits right. Because all women are shaped differently and aren't a perfect ratio of butt to height, but a lot of styles don't accommodate anything other than a strict narrow formula. You know what, one of my most comfortable pairs of jeans as a teenager was a pair of boy's jeans. It's now actually a thing these days for guys to get girls jeans. They often have a better range of colors and styles, and some cis guys find they fit better than the dude cuts too. They don't like the lack of decent pockets either.

Tl;dr: gendered clothes don't fit a lot of people right anyway.