What is the ratio of badges given for comments and those given for posts? Could that give insight into the lack of strong correlation?
As I stated in the article, I think understanding the difference between ratio of badges given for comments as opposed to posts is important to quantify. In the future, I would like to collect that data and run another analysis. However, it won't explain the lack of strong correlation since comment rating is also not significantly correlated with popularity rank or follower total. I think the cause of lack of strong correlation is from not taking into consideration the number of times someone has commented. Those that comment infrequently and/or those who have just joined the site are being ranked high because they have a small sample size of comments. (I probably explain it better in point 3 of the discussion)
I think to add another dimension--and this could only be done with some greater ability to access data, which I don't think we have (do we, mk?)--is to look at the comment score in the top of a thread, and the number of threads started. The comment score of heavy users is probably driven down by long conversations between users. No one is going to upvote a comment that's the 10th response about some topic that's only interesting to the two parties involved. The number of thread started, and the average score of those initial comments could say a lot about how engaged the user is, as well as how much the user contributes to getting others involved.
Definitely agree with this. There is a lot of data that is just publicly available, but it would just take a long time to organize. In the future, I hope that those who are interested, can organize it all and hopefully offer mk and thenewgreen some useful information to make Hubski even better!
Perhaps a metric that is closer to the sum of upvotes with some kind of weight on currency rather than the mean upvotes a users comments received (I assume that is what we have) would solve the issue . . . if there is one. I would be thrilled to find out top commentary does not come from the same people as best stories. Division of labor and all that.