As a youth, he sold bread and canned goods door to door and a coal worker carrying bags of coal to support his family during the Japanese Occupation. As he was fearful of the Japanese soldiers, he had to learn to be streetwise, and to size people up quickly in order to survive, and make money to support his family. He continued his education at St Joseph's Institution. Goh also won a scholarship to study at the prestigious Raffles College (a predecessor of the National University of Singapore) and graduated with a BA in English.