Griffes was born in Elmira, New York. He had early piano lessons with his sister Katherine and later studied piano with Mary Selena Broughton, who taught at Elmira College. Broughton had a profound impact on his personal and musical development. After early studies on piano and organ in his home town, on recommendation of Broughton, he went to Berlin to study with pianist Ernst Jedliczka and Gottfried Galston at the Stern Conservatory. Although recognised as a performer, Griffes grew more interested in composition. Despite being advised against it by Broughton, he left the conservatory and was briefly taught by composer Engelbert Humperdinck. During his time in Berlin he composed several German songs and the Symphonische Phantasie for orchestra.