During Kelly's time as sheriff, his wife and children died and he left New York for an extended overseas trip. After nearly three years, he returned in 1871 and aided Charles O'Conor, Samuel J. Tilden, and their associates in the struggle against the Tweed ring, and Kelly cooperated with Tilden in reorganizing the political machine. The Tweed ring scandal destroyed the old Tammany leadership and shattered the democracy. It was a time of corruption and deceitful politics. Kelly was away and was seemingly untouched by the corruption and so was able to assume the leadership of Tammany Hall.