Barn Elms replaced the earlier one occupied by Sir Francis Walsingham, "Elizabeth's Spymaster". Queen Elizabeth I would visit Barn Elms to see her Spymaster. Barn Elms features in English literary history from the time the royalist poet Abraham Cowley moved to the house belonging to John Cartwright in 1663. In the 1660s Barn Elms became a fashionable destination for boating picnics. Samuel Pepys, who arranged many a Sunday afternoon or moonlit evening boating party to Barn Elms himself, recorded that on 26 May 1667: "I walked the length of the Elmes, and with great pleasure saw some gallant ladies and people come
Thomas Walsingham (literary patron) with Poley present, purportedly in a dispute over an unpaid debt. Poley later became an important, secret intermediary in clandestine arrangements for installing Elizabeth's putative successor, King James. Francis Walsingham made use of his young relative as early as October 1580, when he appointed him as one of the trusted couriers between the English court and the queen's ambassador in France. In August 1581 Thomas accompanied Sir Francis to Paris on a delicate diplomatic mission connected with the proposed marriage between Elizabeth and the French king's brother, Francis, Duke of Anjou. In 1596 he was appointed Justice of the Peace