On October 3, 1861, Ayres was appointed chief of artillery for William F. "Baldy" Smith's division (later designated the 2nd Division of the VI Corps) of the Army of the Potomac. He served in that position in the Peninsula Campaign, the Seven Days Battles, and at the Battle of Antietam. Just before the Battle of Fredericksburg, he was promoted to chief of artillery of the VI Corps as a brigadier general, as of November 29, 1862. At Fredericksburg he commanded the corps artillery stationed across the Rappahannock River on Falmouth Heights.