Kelly was apprenticed to the mason's trade, and engaged in business for himself at the age of 21. Kelly, in response to anti-Catholic sentiment, was driven to politics and became a champion of Catholic and immigrant causes in the 1840s. Kelly joined the influential Tammany Society and the next year he was elected alderman; until his death, he was active as a Democratic politician. From 1855 to 1858, he served in Congress, the only Catholic in the House of Representatives in that period of Know Nothing ascendency. Kelly was later elected Sheriff of the County of New York and served from 1859 to 1861 and again from 1865 to 1867.