Ted Baillieu is married to Robyn and they have three children: Martha, Eleanor and Robert. He is a supporter of the Geelong Football Club, where he is a joint convener of We Are Geelong Supporters (WAGS). Baillieu is a regular Sunday morning swimmer with the "Brighton Icebergs" including former Mayor of Brighton John Locco. He regularly enters the Pier to Pub swim organised by the Lorne Surf Life Saving Club, and in 1989 he Co-founded and has swum in all the Portsea Surf Life Saving Club's Pier to Perignon event, over the 4.5 km course from Sorrento Pier to Portsea Pier. He also plays golf and basketball.