Ma studied at the Juilliard School of Music. His teacher was Leonard Rose. He went to Columbia University and then Harvard University. At one time he had doubts about whether he should continue his studies, but in the 1970s he was inspired by the recordings of Pablo Casals.