Kelly's trips did not distract him from projects at home. On 9 October 1871, he married Louisa Bliss, daughter of William Blowers Bliss, a prominent judge from Nova Scotia. Kelly took on leadership of the completion of the Cathedral of St. John the Baptist (not to be confused with the Basilica of St. John The Baptist). The Cathedral's nave, finished in 1850, had been serving as the entire church. The cathedral was finished in 1885, but burned down in the Great Fire of 1892. Kelly organized its reconstruction. The Cathedral bears a memorial window dedicated to his memory.