Phillips was born Holly Michelle Gilliam on June 4, 1944, in Long Beach, California, the second child of Joyce Leone (née Poole; November 2, 1919 – January 18, 1950), a Canadian-born accountant, and Gardner Burnett Gilliam (July 18, 1914 – March 22, 1996), a merchant mariner from San Diego. She had one older sister, Russell Ann (April 17, 1943 – December 10, 2018). Phillips's paternal grandfather, Marcus Gilliam, was from Walla Walla, Washington and worked as a miner and hotelier in Erie, Pennsylvania. Gilliam County, Oregon takes its name from her paternal ancestors. Her mother suffered heart problems stemming from a childhood bout with rheumatic fever, including subacute endocarditis, and died of a related brain hemorrhage when Phillips was five years old. Reflecting on her mother's illness, Phillips said: "They knew it was only a matter of time... She would lie on the couch in the evenings, listening as my father read to her. One night, after my sister and I had been put to bed, my mother just raised her head, fell unconscious on the couch, and that was it."