 Talley was born in Brandywine Hundred, Delaware to Reverend Lewis S. and Priscilla (Clark) Talley.He attended the Forwood School and graduated from Professor Sudley's Academy in Wilmington, Delaware in 1853.When the civil war broke out, Talley sold his newspaper and organized a company mainly from Delaware County, Pennsylvania known as the Rockdale Rifle Guards.