Confederate President Jefferson Davis appointed Randolph as Secretary of War on March 18, 1862, and he took office on March 24, 1862. Randolph helped reform the department, improving procurement and writing a conscription law similar to one he had created for Virginia. He strengthened the Confederacy's western and southern defenses, but came into conflict with Jefferson Davis. He also was involved in a controversy over the use of hidden shells, which Union troops found upon capturing Yorktown; Randolph argued the explosive devices contravened the laws of civilized warfare but were acceptable if left on a parapet of a fort to prevent its capture or allow defenders to retreat more safely. With weakening health due to tuberculosis (TB), he resigned on November 17, 1862.