ctrl-f Milgram, was not disappointed.
Dan Ariely has an entire book about this, which takes the form of thought experiments followed by real experiments followed by results. From a behavioral economics standpoint, it's not a desire to fit in that causes the most harm, it's the belief that we can get away with a little bit because everyone else is doing it. We know right from wrong but it's a relative scale. We know we can't step over the line a whole lot more than the next guy or we'll get creamed. It's not so much a desire to fit in as it's a desire to not be found out.
This doesn't mean people have no morals. It means that our moral compasses are set just a little east of true north where we feel true north is what we're supposed to do. Human misbehavior is almost exclusively constrained within that margin of error because nobody else thinks it's misbehavior. "Wrongdoing" is the stuff that society proscribes and despite Milgram's (outlier, discredited) study, there are no studies indicating that humans will blindly follow other human beings they know to be committing violations of the social code.