This is a semantic problem. I say 'We used our self-moderation tools to shut the door on a troll and ignore him. We ignored him and he went away.' You say 'We didn't ignore him, we reacted to offensive things and took steps to minimize our contact with offensive things which is not ignoring him.' Freedom of speech lets you knock on someones door and say 'I want to talk about these things, this is what I believe about these things.' It protects a person from being violently assaulted or imprisoned for speaking. It doesn't protect them from someone closing the door. Refusing to engage, making it impossible for the troll in question to engage is the functional equivalent of ignoring it. I can't say who the 'right' targets are for violence. But I can say in the affirmative that giving Milo Yiannopoulos of all people the moral high ground is asking for more trouble. Engaging with him in any manner is asking for more trouble.