Part of the problem is that he promises things that are a physical impossibility, and those of us who can do basic math say, "You know that's impossible, right?" And then every yells at you for having no imagination. I know a lot of people even within the auto industry who think he shits gold. I am not one of those people. I think the thing he has done effectively is to revive the market for electric cars, which should have a place in the world, albeit not at 100% market share like Europe and China are proposing. I know far, far less about SpaceX and space travel generally than I do about Tesla and the auto industry, so I don't consider myself in a position to evaluate any of his rocket claims. I find it eternally hilarious, however, being a veteran of the manufacturing world, that Tesla seemed to lack any conception that making cars is hard. I'm retired form Tesla bashing, however, so I'll shut up now.
SpaceX exists because Energia wouldn't sell him boosters so he set out to duplicate the Soviet rocket program. ULA is all about "as good as it can be on a cost-plus contract" while SpaceX is all about "Good enough on a turnkey contract." I'm not saying the the team at SpaceX is incapable of the standards of ULA. But as SpaceX never had the luxury of a cost-plus military development cycle, they've had to value-engineer everything they do. The end result is craft well-capable of space at a different price point than The Usual Suspects.