Guy maybe a total ahole but allowing violent protesters to deny him his right to speech, and the right of those attending to listen sets a really bad precedent.

That precedent had already been set. Ever heard of Selma, Alabama? Or MLK?

And he wasn't denied his right to speak. He spoke. People didn't like what he said, and they drove him away.

So what?

You are missing the point of the comic: Milo is a dick. Nobody has to give him a stage to be a dick from. Nobody has to listen to him being a dick. If he had any balls or integrity, he'd go to Speakers Corner and make use of the rights provided to him.

But he's just a troll. Having integrity and engaging honestly doesn't interest him. He just pokes the beehive so he can complain about getting stung.

Same model as the Westboro Baptist Church. They are all lawyers who have found the simplest way to make money is to be a dick in public, and when someone reacts, you sue them. That's their entire reason for existence: to be such colossal dicks that people get upset, and then use the legal system to threaten them until they pay up. Simple extortion wrapped in the sheep's clothing of "free speech."