Barn Elms replaced the earlier one occupied by Sir Francis Walsingham, "Elizabeth's Spymaster". Queen Elizabeth I would visit Barn Elms to see her Spymaster. Barn Elms features in English literary history from the time the royalist poet Abraham Cowley moved to the house belonging to John Cartwright in 1663. In the 1660s Barn Elms became a fashionable destination for boating picnics. Samuel Pepys, who arranged many a Sunday afternoon or moonlit evening boating party to Barn Elms himself, recorded that on 26 May 1667: "I walked the length of the Elmes, and with great pleasure saw some gallant ladies and people come
Francis Walsingham has already overwhelmed the brethren and neighbours in France and Flanders embrace us which be left in such sort as we shall not be able to escape." Walsingham tracked down Catholic priests in England and supposed conspirators by employing informers, and intercepting correspondence. Walsingham's staff in England included the cryptographer Thomas Phelippes, who was an expert in deciphering letters and forgery, and Arthur Gregory, who was skilled at breaking and repairing seals without detection. In May 1582, letters from the Spanish ambassador in England, Bernardino de Mendoza, to contacts in Scotland were found on a messenger by Sir John Forster,