She was born Pauline Potter at 10 rue Octave Feuillet in the Paris neighborhood of Passy, to wealthy expatriate American parents of Protestant background. Her mother was Gwendolen Cary, a great-grand-niece of Thomas Jefferson and a distant cousin of Britain's Lords Falkland and Cary. Her father was Francis Hunter Potter, a playboy who was a grandson of Alonzo Potter, an Episcopal Bishop of Pennsylvania, and a nephew and great-nephew of successive Episcopal bishops of New York, Horatio Potter and Henry Codman Potter.