In 1842, Conroy settled at his family home in Arborfield Hall near Reading, Berkshire and became a gentleman farmer, winning prizes for his pig breeding. He founded the Montgomery Regiment of Militia in 1849. Despite his pensions and ownership of properties and lead mines in Wales, Conroy was in substantial debt when he died on 2 March 1854 at Arborfield. His eldest son Edward succeeded him to the baronetcy. After his death, the Duchess of Kent finally agreed to open her financial accounts and acknowledged that significant funds were missing. She was moved to admit that Conroy had swindled her while at the same time hurting her relationship with Victoria for his own benefit. A rapprochement followed between mother and daughter.