Sullivan was born to a Swiss-born mother, Andrienne List (who had emigrated to Boston from Geneva with her parents and two siblings, Jenny, b. 1836, and Jules, b. 1841) and an Irish-born father, Patrick Sullivan. Both had immigrated to the United States in the late 1840s. He learned that he could both graduate from high school a year early and bypass the first two years at the Massachusetts Institute of Technology by passing a series of examinations. Entering MIT at the age of sixteen, Sullivan studied architecture there briefly. After one year of study, he moved to Philadelphia and took a job with architect Frank Furness.