Born on a farm in Howard County, Maryland, he became a railroad executive before branching out into coal mining and banking as founder of the Potomac and Piedmont Coal and Railway Company. Davis won election to both houses of the West Virginia Legislature before serving as U.S. Senator from 1871 to 1883. His younger brother, Thomas Beall Davis, also served in Congress. After his tenure in the Senate ended, Davis continued to grow his business interests. In partnership with son-in-law, Stephen Benton Elkins, Davis created the Davis Coal and Coke Company and led it to become one of the largest coal companies in the world.