Bayley returned to Hartford as a resident graduate to begin his medical studies. However, he subsequently changed his mind and began studying under the Rev. Samuel Farmar Jarvis to enter the Episcopal ministry. It is not certain what was the determining factor in this change in the course of his studies. In his autobiographical sketch, Bayley stated simply: "Studied medicine one year & then Theology with the Revd. Dr. Saml. Farmar Jarvis, at Middletown." He spent several happy, profitable years there studying under an eminent clergyman who was an authority on ecclesiastical history and antiquities, and allowed his students free access to his personal library of 10,000 volumes. He became acquainted with "the elegant Oxford edition of the Fathers and the more erudite French and Italian editions of the same and other important works" and "waded through the literature of the middle ages," such as the works of Peter of Blois and Vincent of Beauvais which served to remove from his mental vision the scales of prejudice against the mis-named dark ages.