While working at the Fall River shipyard during World War II, Townsend was arrested on January 29, 1943, for participating in an "unnatural and lascivious act". The "Mid-Town Journal" headline reported, "Beacon Hill 'Twilight' Man Member of Queer Love Cult Seduced Young Man". He did not deny it, and was sentenced to eighteen months in the Massachusetts House of Corrections on Deer Island. No one in his influential family applied any pressure to shorten his jail time. A month later he was officially stricken from both the New York and Boston Social Registers.