At the start of the Civil War Stoneman was in command of Fort Brown, Texas, and refused the order of Maj. Gen. David E. Twiggs, a southern sympathizer, to surrender to the newly established Confederate authorities there, escaping to the north with most of his command. Returning east, he was reassigned to the 1st US Cavalry and promoted to major on May 9, 1861. Stoneman then served as adjutant to General George McClellan during his campaign in Western Virginia during the summer. After McClellan became commander of the newly-formed Army of the Potomac, he assigned Stoneman as his chief of cavalry; Stoneman was promoted to brigadier general of volunteers on August 13. Stoneman had a difficult relationship with McClellan, who did not understand the proper use of cavalry in warfare, relegating it to assignment in small units to infantry brigades.