Choy was born in a Hakka family in Kudat, North Borneo (now Sabah). Her great-grandparents first came to Kudat from Hong Kong to assist German missionaries in their work. The eldest of 11 children, Choy's father worked as a civil servant after completing his early education in China with some English education in North Borneo, where he married the daughter of a priest. He transferred to work in Jesselton (now Kota Kinabalu) and was later promoted to District Officer and he moved to Kalimantan.