 Charles Le Roy was the youngest of the four sons of French watchmaker Julien Le Roy (1686–1759).He had three brothers, the watchmaker Pierre Le Roy (1717–1785), the physicist Jean-Baptiste Le Roy and the architect Julien-David Le Roy (1724–1803).His paternal uncle Pierre Le Roy (1687-1762) was also a watchmaker.