Ima Hogg was born in Mineola, Texas in 1882 to Jim Hogg and Sarah Ann "Sallie" Stinson. She was the second of four children, including brothers William Clifford Hogg (1875–1930), Michael Hogg (1885–1941), and Thomas Elisha Hogg (1887–1949). The Hogg family had long been active in public service. Her great-grandfather, Thomas Hogg, served in the state legislatures of Georgia, Alabama, and Mississippi. Her grandfather, Joseph Lewis Hogg, served in the Congress of the Republic of Texas and helped to write the Texas State Constitution. At the time of her birth, Hogg's father was the district attorney of the Seventh District in Texas. His term expired in 1884, and the family moved to Tyler, where he practiced law. Two years later, Jim Hogg was elected Texas Attorney General and the Hogg family moved to the state capital, Austin, where Ima attended kindergarten. When Jim Hogg was elected the first native-born Governor of Texas four years later, Ima accompanied her mother and elder brother to the swearing-in ceremony and inauguration ball in January 1891, thus witnessing the first inauguration in the newly erected Texas State Capitol. The family moved into the Governor's Mansion. Built in 1855, the building was in poor condition, with cracked walls and dilapidated furnishings. Ima and her siblings were expected to help renovate the building to a liveable state—she was required, among other things, to pry chewing gum from the furniture and door moldings.