He travelled to British Columbia in 1862 during the gold rush there. In 1864, he returned to Martintown, where he married Margaret McDougall.
Rayside was born in Montreal in 1836, the son of Scottish immigrants, and was raised by his grandparents in Martintown, Upper Canada. He learned the trade of carpentry and became a millwright.