Burns was born in Burlington, New Jersey, of Scottish ancestry; his father claimed a relationship with poet Robert Burns. He served as an enlisted man in the War of 1812, fighting in numerous battles, including Lundy's Lane, and volunteered for both the Mexican–American War and the Civil War. He was rejected for combat duty in the latter war due to his advanced age, but he served as a teamster in support of the Union Army. He was sent home against his will to Gettysburg, Pennsylvania, where he was named constable. During Confederate Maj. Gen. Jubal A. Early's brief occupation of Gettysburg on June 26, 1863, Burns was jailed for his adamant assertion of civil authority in resisting. As the Confederates departed, Burns was released from jail and arrested some of the Confederate stragglers, continuing his opposition to the invading army until he was relieved by Federal cavalry under Brig. Gen. John Buford.