O'Rahilly was born in Ballylongford, County Kerry to Richard Rahilly, a grocer, and Ellen Rahilly (née Mangan). He was educated in Clongowes Wood College (1890–1893). He had two siblings who lived to adulthood, Mary Ellen "Nell" Humphreys (née Rahilly) and Anno O'Rahilly, both of whom were active in the Irish revolutionary period. As an adult, he became a republican and a language enthusiast. He joined the Gaelic League and became a member of "An Coiste Gnotha", its governing body. He was well travelled, spending at least a decade in the United States and in Europe before settling in Dublin.