Fraser went to Canada with the British forces in the French and Indian War and took part in the Siege of Louisbourg. He was promoted to Captain before taking part in the Battle of Quebec in 1759. At that battle, he was in James Wolfe's boat crossing St Lawrence. It was his reply, in French through the fog, that enabled the party to sneak ashore before ascending to the Plains of Abraham. He is not the Simon Fraser depicted in the famous painting The Death of General Wolfe.