After Speer's indictment and imprisonment for war crimes, Wolters stood by him. In addition to receiving and organizing Speer's clandestine notes from Spandau, which later served as the basis of his best-selling books of memoirs, Wolters quietly raised money for Speer. These funds were used to support Speer's family and for other purposes, according to directions which Wolters received from his former superior. Following Speer's release in 1966, their friendship gradually deteriorated, until the two men became so embittered that Wolters allowed papers demonstrating Speer's knowledge of the persecution of the Jews to become public in 1980.