Folger resigned from the State Senate in 1869, having been appointed by President Ulysses S. Grant to Assistant United States Treasurer in New York City. The following year, Folger was elected one of the first judges of the re-organized New York Court of Appeals. After the death of Sanford E. Church, Folger was appointed Chief Judge by Governor Alonzo B. Cornell on May 20, 1880, to fill the vacancy temporarily. In November he was elected to a full 14-year term as Chief Judge.