Ho is of Chinese, Hawaiian, Portuguese, Dutch, and German descent. He was born in the small Honolulu neighborhood of Kakaako. Ho grew up in Kāneohe on island of Oahu. He was a graduate of the Kamehameha Schools in 1949. He attended Springfield College on a football scholarship in 1950. Ho left Springfield and returned home to earn a bachelor's degree in sociology at University of Hawai'i in 1953. In 1954, Ho entered the United States Air Force. He did his basic training at Keesler AFB, Mississippi. Ho spent time flying fighter jets in both Texas and Hawaii. When he was sent to Hamilton AFB, California, he went to the local town of Concord and bought an electronic keyboard from a music store. He was known to say, "That's when it all started."