During the Civil War, he organized the 126th New York Infantry in August 1862 and became its first colonel. His regiment was among the troops defending Harpers Ferry against Joseph B. Kershaw's Confederate brigade of Maj. Gen. Lafayette McLaws's division during the Maryland Campaign. He was severely wounded with a gunshot through his lower jaw in fighting on Maryland Heights during the Battle of Harpers Ferry. Sherrill was captured and later paroled. The wound never healed, but he temporarily rejoined his regiment at Union Mills, Virginia, in October 1862. After a furlough for further recuperation, he returned for duty on January 27, 1863.