This is more of my point. I don't think that authorities will show up at your door for making random searches. But if someone wants to mess with you and knows you did this, you have a bit more explaining to do than if you hadn't.

Any cast of suspicion is another obstacle you'd have to overcome to have people dismiss the claim, regardless of how unwarranted the claim may seem.