On the final day of his breakfast show in November 2004, he announced that his partner Melanie Mossman was pregnant and that the next year he would be leaving his media career to become a full-time dad. He returned within two years to present another breakfast radio show. He married Mel on 28 January 2006 in the Great Hall at Sydney University, where guests were asked to donate to charities in lieu of gifts. They have two daughters, Elaquare (Ellie) and Olivia. When Elaquare was six months old, Spencer organised for a fly to be named after her, Fijian fly "A. Amblypsilopus elaquarae" ("Elaquare's Amblypsilopus"). This was through the Australian Museum's project to seek funding by giving new animal species namesakes for a donation.