I'm glad to get a perspective from the other side of the discussion! I see what you're saying and I do agree. It wasn't so very long ago that math, science, the arts, religion and magic were all one and the same and it's easy to forget that shared history. I also think that the way that knowledge is divided into subjects is somewhat harmful to public education. For example, some understanding of earth sciences and chemistry is very useful in arts such as sculpting and painting and of course, history ties those subjects together in yet another way. The ideas of revelation, understanding, mastery and transcendence are common to all disciplines, so I have a hard time understanding why practitioners of those many varied areas of endeavor are sometimes quick to dismiss the works and efforts of others. I could be very wrong, but I suspect that while we are now in an era where the artistic and scientific are fairly well delineated, that in time the two will once again mutually inhabit certain spaces in the human experience.The formulas and proofs used often do provide satisfaction to the practicing mathematician, in the same way an inspiring painting, photo, musical piece, or comedic bit provides pleasure while inspiring new work. However, I don't think they're "beautiful" in and of themselves. Rather, it's the new perspective that comes with a deeper understanding of the underlying structures you come across.
It's the "ah, look at that" aspect that can sometimes be very striking and even emotional. Made all the more powerful by its timelessness and transcension of humanity.