One of my main jobs atm is removing comment spam that slips through which annoys people. I don't expect that I will be removing less, but that it will annoy people less.Effectively you're getting rid of our ability to control comment spam.
This is not a healthy approach in my opinion. The site doesn't run without you and you're opting to increase your toil and drudgery for little added benefit. I would much rather your main job to be doing stuff that is fun. Off the top of my head, what if new accounts had a "spam" button as well as "promote" and if "spam" gets hit a half-dozen times without "promote" getting hit, the comment gets removed? And if an account gets, say, ten comments removed it shows up wherever you look for such stuff to give you the option to remove the user?