Converse was born in Newton, Massachusetts, the son of Edmund Winchester and Charlotte Augusta (Shepherd) Converse. His father was a successful merchant, and president of the National Tube Works and the Conanicut Mills. Frederick Converse's higher education was at Harvard College, where he came under the influence of the composer John K. Paine. Converse had already received instruction in piano playing, and the study of musical theory was a most important part of his college course. Upon his graduation in 1893, his violin sonata (op. 1) was performed and won him highest honors in music.