Conroy was born in Wales to Anglo-Irish parents. In 1817, after holding several ranks in the army, he became the equerry of Prince Edward, Duke of Kent and Strathearn. Edward died two years later, leaving a widow and infant daughter. Holding the position of comptroller of the Duchess of Kent's household for the next nineteen years, Conroy also acted as her confidant and political agent, among other roles. Together, they designed the Kensington System, an elaborate and strict system of rules for the upbringing of young Victoria, designed to render her weak-willed and utterly dependent upon them in the hope of allowing them one day to wield power through her.