Definition Holy:  dedicated or consecrated to God or a religious purpose; sacred: 
"the Holy Bible" 
According to this definition, Church is Holy.  What people do in church, mainly worshipping God, it is holy place!  
A place hold the energy of the what people did in that place.  
St. Paul Cathedral can hold 4000 worshipers and few thousand people pray together will generate a tremendous spiritual energy for good and it's walls and ceilings hold that spiritual energy.    That is one of big reasons that even a non believer could feel the presence of God in these holy places.