Following a few years in England, Holder returned to New England, settling in Newport, Rhode Island. He had purchased Patience Island from Roger Williams, and given it to his wife as a wedding gift. He traveled extensively, particularly in 1663, but was likely back in England in 1666 when his son was born in Gloucestershire. In 1672 Holder was with George Fox in New York, during the visit Fox made to New England. In 1673 his name appeared on a list of freeman in Newport.