While still studying with Dr. Jarvis, Bayley was ordained a deacon in Christ Church on October 3, 1839. Shortly afterwards, he received a letter from his cousin, Emma Craig, informing him that through the influence of his family, he was to have St. Andrew's Episcopal Church in Harlem while the Rev. Abram B. Hart was absent due to illness. Bayley served there through the winter of 1839–1840, and happily spent the following summer in the discharge of ministerial duties in Zion Church at Avon. After Mr. Hart's continued illness forced him to resign the following September as rector of St. Andrew's, Harlem, an entry in the parish register recorded that on October 19, 1840, "the Rev. James Roosevelt Bayley, deacon, received and accepted a call to the rectorship." On February 14, 1841, he was ordained in St. Andrew's Church by Bishop Benjamin T. Onderdonk, and "admitted to Priest's Orders."