Born at Saffron Walden in Essex, Smith was the second son of John Smith of Walden by Agnes, daughter of John Charnock of Lancashire. The Smiths of Essex are said to be descendants of Sir Roger de Clarendon, an illegitimate son of the Black Prince. He was educated at Queens' College, Cambridge, where he became a Fellow in 1530, and in 1533 was appointed a public reader or professor. He lectured in the schools on natural philosophy, and on Greek in his own College. In 1540 Smith went abroad, and, after studying in France and Italy and taking a degree in law at the University of Padua, returned to Cambridge in 1542.