Osler descended from a British family with a long naval tradition. Edmund's great grandfather, Edward Osler, was variously described as either a merchant seaman or a pirate. One of Edmund's uncles served as a medical officer in the Navy and wrote the "Life of Lord Exmouth" and the poem "The Voyage". Osler’s father was Featherstone Lake Osler (1805–1895). Featherstone was the son of a ship owner from Falmouth, Cornwall. Featherstone formerly served in the Royal Navy as a Lieutenant on the H.M.S. Victory. In 1831 Featherstone Osler was invited to serve on H.M.S. Beagle as the science officer on Charles Darwin's historic voyage to the Galápagos Islands, but turned it down as his father was dying. Featherstone retired from the navy in 1837 and emigrated to Canada. Featherstone married Ellen Free Pickton and they had four sons, Edmund was their youngest. In Canada, Featherstone became an Anglican Reverend.