What do you mean by society? Every individual?

The society is the overall trends and behaviors of any group. An indavidual is a single person and their actions, who may belong to a sub-group of society.

How small can that subset get before it's not "society" anymore?

If you judge a person, as an indavidual, by any measure but the status of that single indavidual, you've crossed the line. If your actions of judging some group results in you judging individuals not as individuals than you've crossed the line. You do this when you attempt to judge two people differently even if their individual actions had the same level of impact.

And what does judge mean?

Any action or consideration of a person. It can mean a whole range of things. Essentially, any attempt to put pressure on a person to change their behavior or tendencies.

Why can't a group of people "judge" an individual?

A group of people can judge an individual for their individual actions. However, society as a whole shouldn't begin to put pressure on select individuals for problems that only arise due to the behavior of the trends of a large group of people acting that way. You can do things like put pressure on "people who do X" because you are still judging people based on their actions, and applying that to all of society. However, judging only people who are X and do X is wrong.

There are societal trends that only arise when you look at society at the large scale, but don't necessarily appear on the individual scale. These trends should be prevented by actions that effect everyone in society, rather than targeting specific people.