At some point during this period, the building was damaged by fire. There are several stories connected with this incident. One holds that the club set fire to the building when William Conolly's son refused to renew the lease on the lodge. An alternative story claims the club members did it to give the building a hellish appearance. Another story recounts that, following a black mass, a footman spilled a drink on "Burn-Chapel" Whaley's coat. Whaley retaliated by pouring brandy over the man and setting him alight. The fire spread around the building and killed many members. Following the fire, the club relocated further down the hill to Killakee Stewards House. However, the club's activities declined after this incident.