Although Gan had little education, he realised its value as he prospered and became wealthy. His dream to build a school for the poor which taught both English and Chinese (Hokkien, as Mandarin was not commonly used in Southeast Asia then) was fulfilled in 1885, when he built the "Anglo-Chinese Free School" for boys in some shophouses in Telok Ayer Street (not related to the Anglo-Chinese School founded a year later by Bishop W.F. Oldham). In 1923, it was renamed to Gan Eng Seng School (GESS) in his honour.