Reynolds was born in Montgomery County, Maryland. He was appointed at age eighteen as a cadet to the Revenue Cutter Service School of Instruction at Curtis Bay, Maryland, on 24 May 1878 and after two years training was commissioned as a third lieutenant on 17 July 1880. He was in the second class graduated from the School of Instruction. As a member of the crew of USRC "Corwin", in 1881, he participated in the search for the missing exploration steamer USS "Jeannette" along the northern coast of Siberia using a sled team. On 12 August 1881 he was a member of the expedition which claimed Wrangel Island for the United States while searching for "Jeannette".