Akiyoshi was born in Liaoyang, Manchuria to Japanese emigrants. She was the youngest of four sisters. In 1945, after World War II, Akiyoshi's family lost their home and returned to Japan, settling in Beppu. A local record collector introduced her to jazz by playing a record of Teddy Wilson playing "Sweet Lorraine". She immediately loved the sound and began to study jazz. In 1952, during a tour of Japan, pianist Oscar Peterson discovered her playing in a club on the Ginza. Peterson was impressed and convinced record producer Norman Granz to record her. In 1953, under Granz's direction, she recorded her first album with Peterson's rhythm section: Herb Ellis on guitar, Ray Brown on double bass, and J. C. Heard on drums. The album was released with the title "Toshiko's Piano" in the U.S. and "Amazing Toshiko Akiyoshi" in Japan.