Lee was born in Hawick, Scotland, 9 August 1831. At age 12 he created his first gun. He accidentally blew himself up with gunpowder. He also accidentally shot himself, twice. His family was surprised he reached adulthood. He was obsessed with guns, but he was apprenticed to a clockmaker. His family migrated to Canada. Lee worked making and repairing clocks and watches. In 1850 he opened his own shop where he could work on rifles in his spare time.