Heilmann was born in San Francisco in 1894. His father, Richard W. Heilmann, was born in 1860 at Sacramento, California, the son of an Irish mother and a father from Saxony. His mother, Mary (McVoy) Heilmann, sometimes referred to as Mollie, was born in 1864 in County Roscommon, Ireland. Harry was the couple's fourth surviving child, having an older sister, May, and two older brothers, Richard, Jr., and Walter. Heilmann's father operated a soap business, ran as a Republican for the office of Supervisor of San Francisco's Ninth Ward in 1896, and died in June 1897 at age 36. As a boy in the 1890s and continuing at least through the time of the 1900 Census, Heilmann lived with his family at 708A Brannan Street in what is now the South of Market district of San Francisco.