Carlson devoted his wealth to philanthropic purposes. He donated over $150 million to charitable causes and was an active supporter of the NAACP. Carlson's wife Dorris got him interested in Hinduism, particularly the ancient texts known as the Vedanta, as well as in Zen Buddhism. They hosted Buddhist meetings, with meditation, at their home. After reading Philip Kapleau's book "The Three Pillars of Zen", Dorris invited Kapleau to join their meditation group; in June 1966, they provided the funding that allowed Kapleau to start the Rochester Zen Center. Dorris paid for of land that became Dai Bosatsu Zendo Kongo-ji, a Zen monastery in the Catskill Mountains of New York led by Eido Tai Shimano. Carlson had purchased a New York City carriage house for use by Shimano; he died four days after it was dedicated. Carlson is still commemorated in special services by Shimano; his dharma name, "Daitokuin Zenshin Carlson Koji", is mentioned.