In a society, if one person or one business frequently takes advantage of others, is it not true that this person will eventually not be able to find people to transact with?
There's a lot of social detritus from this type of behavior though. Berwick lost a lot of money, but he's a rich, established investor engaging in out of country deals. Imagine if Madoff existed within this libertarian structure--all those savings would have been withheld from the scammees for the express reason "you should have known better."
There are a lot of really exploitative business practices in this world that's populated with not too many business savvy individuals. When was the last time a friend tried to sell you on a pyramid scheme? There needs to be mandated adjudication/protection because not everyone can be an expert all the time. This adjudication needs to be impartial as well, and it's my belief (not fact) that a private third party adjudication system is too open to corruption and collusion because they, and no one else, would be the party determining guilt in business transactions.