I finally found that Duluth Trading Company has consistently-sized clothing, because they make most of it themselves. After one shipment where I ordered three different sizes, found the one that fit me, and returned the others, it's a no-brainer to order from them now. I know my size, and their clothing is consistent.
Almost every other retailer simply buys clothes from a variety of bulk manufacturers, and slap their label on it. So sizes are all over the board, since every mfg has their own idea of sizes.
So yeah. I buy from Duluth and Zappos, because I know the sizing is good. And when it isn't, it is dead-easy to return the stuff.