From the 1880s, Burns was based at a property known as Gowan Brae, North of Parramatta. He retired from government and business activities due to ill health in 1908, and lived at Gowan Brae from that time on. Shortly afterwards, he approached the Presbyterian Church of Australia, suggesting that they establish a Presbyterian home for children. In 1910 he endowed some of the Gowan Brae property, to establish the Presbyterian Homes for Children (later renamed Burnside and currently known as UnitingCare Burnside) and was chairman of its board for ten years. It became his passion in his "retired" years, as he made numerous contributions towards new homes and the children. His business and government connections ensured that donations from other sources were also forthcoming, with houses named after sponsors and officially opened by dignitaries.