After finishing schooling, Carroll worked as a student lawyer for the law office of Brown and Brune in Baltimore. He was admitted to the bar in 1851. Carroll practiced law in Maryland from 1854 until 1858. He ran as a Howard County Democratic candidate for the state General Assembly in 1854, (shortly after the separation of the former Howard or Western District of Anne Arundel County and the "erection"/establishment of Howard as the 22nd of the state's 23 counties), however losing to his opponent from the newly-dominant "Know Nothing" Party (also known as the American Party) during the political crises of the 1850s. Carroll then moved to New York City and while there, accepted a position as deputy clerk and United States Commissioner in the office of the clerk of the United States District Court. He stayed there until 1861, at the outbreak of the Civil War, when he returned to Maryland, where he then remained the rest of his life. When he returned to Maryland, Carroll purchased the "Doughoregan Manor", historic family estate in Howard County, near Ellicott City from his older brother Charles Carroll.