I guess I singled Musk out unfairly. Yes, the interviewer was also a major source of awkwardness. To me the most awkward moments were when Musk made jokes though. Not because his jokes were bad, ("I work, a lot") but the audience didn't really react. Maybe they were too focused on the talk in general? Anyway, I think that added to the awkwardness and it seemed to me like Musk felt some of that too.
Anyway, I suspect that the issue for me likely stems from giving so many speaking tests to students whose first language isn't English and conditioning myself to be critical of the way people speak for professional purposes, that it has bled into my personal life.