His father George was a friend of William Wordsworth. In 1823, on a visit to the Lake District, Henry Taylor made the acquaintance of Robert Southey, and they became friends. Jane Taylor had a first cousin, Isabella Fenwick (1783–1856), whom he introduced to the Wordsworth family. She became a close friend of Wordsworth in later life, as she had been of Taylor up to the time of his marriage. Though Fenwick was not herself a writer, her friendship left an enduring impression on the writings of Taylor and Wordsworth. In his autobiography, Henry Taylor wrote, “There is a good deal of her mind in my writings. I wish there was more; and I wish that she had left her thoughts behind her in writings of her own.”