Rohlfs was born in Brooklyn and studied at the Cooper Union in Manhattan. As a young man, he worked as a stove pattern-maker while pursuing his career as an actor. He received several patents for stove designs, but had limited success as an actor. (Reading a review in 1895 in which a Chicago critic wrote, "His face is comedy, his spindling legs are comedy, and those ponderous double-jointed, floppy hands of his would be two separate and distinct boons to any eccentric comedian" - and Rohlfs was performing a serious role - may have been a turning point in his choice of careers.) He married the successful crime novelist Anna Katharine Green in 1884. After their marriage, he continued his career in the stove industry, and later made another attempt to establish his reputation as an actor. Rohlfs's father-in-law had been prominent in the Republican Party in New York City, and in 1896, Rohlfs participated in public debates in support of William McKinley's presidential campaign.