After William Milne's death in 1822, his four surviving children were sent back to England, to receive further education under the care of Rev. Andrew Reed (1787–1862). Both Robert George and William Charles Milne graduated from Homerton Academy and Marischal College of Aberdeen University. Later, William Charles followed his father's steps and became a missionary to China being sent by London Missionary Society, and a tutor to British translators in China. William Charles died in 1863 and was buried in a Russian cemetery at Beijing. Robert George Milne served as a Dissenting minister in Lancaster, and died in 1882.