Like ButterflyEffect, I also am not religious. From time to time I go to church or other service, but rarely do I feel anything close to spirituality. I am not affected by the words and music as the others around me, nor does it bother me. The closest I've felt to being "spiritual" was at a Taizé service a few years ago. It wasn't spirituality that I felt though. It was peace and tranquility. A feeling of complete relaxation and a closer connection to the world around me. Taizé services are pretty incredible. Nobody preaches and the leader of the service doesn't make themselves obvious. Instead it's a smooth transition between simple, beautiful songs and, if I remember correctly, there was a moment to declare the people you wish to pray for, and then more songs. The service ended in complete silence. I left the church feeling refreshed and at peace. A moment later my brother and our friend started talking about how stupid the whole thing was. Apparently I was the only one who found serenity.
I wouldn't say that they are bad listeners. I believe the problem is more to do with age. At the time I was 17 or 18, they were about 19 years old, non-religious and very cynical. My brother is rarely serious and tries to make everything a joke. It's difficult to be serious about things when it's so much easier to deride them. For them, anything out of the norm is an easy target to get amusement from.