In England Holder was able to confer with Quaker founder George Fox, and was likely involved in the excitement of new religious freedoms emanating from the Declaration of Breda and from the influence of Quaker Edward Burrough with the king. Holder was back in New England by 1663, but continued to travel extensively between New England, England, and the West Indies. He became a resident of Newport, Rhode Island, and twice married to Quaker activists. He was so highly respected in Rhode Island, that during the colony's devastation during King Philip's War, he was one of 16 colonial leaders whose counsel was sought in April 1676 during the very difficult times. Holder had returned to England by the early 1680s, where, following continuing persecution of Quakers, he was arrested on several occasions and imprisoned for more than a year. Following his release in 1685, he lived just a few more years, dying in Gloucestershire in 1688. He is best remembered for his leadership, evangelism, and sufferings in confronting the cruel anti-Quaker laws in the Massachusetts colony.