Goh was born in 1927 a second-generation Singapore-born Hokkien Chinese to a family of five siblings. His father was a clerk in the Yokohama Specie Bank (the former Hong Kong and Shanghai Bank) at Clifford Pier while his mother was a housewife. Three generations had lived on 104 Telok Ayer Street for most of their lives, before relocating to River Valley Road. He had primary education at the Royal English School in the River Valley vicinity, and went on to the St. Joseph's Institution together with his eldest brother, Charlie Goh.