Smith was a judge of the New York City Civil Court from 1975 to 1979 and a justice of the New York State Supreme Court from 1980 to 1986. In 1986, Governor Mario Cuomo appointed Smith to the Appellate Division, First Department, where he served from 1986 to 1992. Smith's best-known opinion for the Court was "People v. LaValle", a 4-to-3 decision holding that New York's death penalty statute was unconstitutional due to the structure of its sentencing procedures.