In Harlem, Bayley had the opportunity, probably for the first time, of conversing with Catholic priests and arguing with them on such controversial questions as papal primacy and the validity of Episcopalian holy orders, which Catholic theologians denied. While acting as rector of St. Andrew's, he often received money to be distributed among the poor, and since there were no poor in his own parish, Bayley directed his ministry to the poor Irish immigrants of the city, who were overwhelmingly Catholic. It was on one of these visits that he met Father Michael Curran, the pastor of St. Paul's, Harlem, who later stated that "he had helped into the Church his neighbor, the Protestant rector of St. Andrew's, the Rev. J. R. Bayley."