He lived for years in a house he had built facing Paddington Green, then a village on the outskirts of London, not far from the house in which he had installed Emma and her mother. He kept Romney's paintings of Emma on the walls of his house until his death. There he indulged his passion for gardening in a large garden provided with glasshouses in which he grew many rare tropical plants, aided by his connection with Banks, and managed to coax "Vanilla planifolia" to flower for the first time under glass, in the winter of 1806–07. His contributions to the herbarium assembled by Sir James Edward Smith are preserved by the Linnaean Society of London. The Australasian genus "Grevillea" is named in his honour.