McPherson was born in 1828 near Clyde, Ohio. He was the eldest son of William and Cynthia (Russell) McPherson who were farmers. To help his family, in 1841 at age 12, McPherson entered an apprenticeship at a general store. He became a clerk for Robert Smith. McPherson was raised and tutored along with Smith's children. His father died in 1847 and Smith paid for McPherson's education at Norwalk Academy. In 1849 Smith arranged an appointment to the United States Military Academy at West Point. McPherson spent much of his time tutoring his John Bell Hood. McPherson graduated in 1853, first in his class.