He was recalled in January 1666 when France joined the Second Anglo-Dutch War as an ally of the Dutch Republic; on 14 November, the diarist Samuel Pepys recorded "Sir George Carteret tells me...my Lord Holles had been with him and wept to think in what a condition we are fallen." In April 1667, he was part of the English delegation sent to negotiate the Treaty of Breda with the Dutch; talks dragged on until the humiliation of the Dutch Raid on the Medway in June led Clarendon to instruct him to agree to terms in order "to calm people's minds" and "free the king from a burden...he is finding hard to bear".