Robert Ward Johnson was born on July 22, 1814, in Scott County, Kentucky, to Benjamin and Matilda ( Williams) Johnson. His father had three brothers who were elected as U.S. Congressmen and the family was politically prominent in the state. His grandfather had acquired thousands of acres of land in the area at the end of the eighteenth century. The family were slaveholders. His siblings included a sister Juliette. His paternal uncles were Richard Johnson, a United States Representative and Senator, and vice president of the United States under Martin Van Buren; and his brothers James Johnson and John Telemachus Johnson, older and younger, respectively, who were each elected as U.S. Representatives from Kentucky.