Hardy was the third son of Major Thomas Bartholomew Price Hardy, from a naval family, and the older brother of William Hardy who had a parallel career as archivist. He was born on 22 May 1804 at Port Royal in Jamaica, and came to England in 1811. He entered the government service on 1 January 1819, obtaining on that date, through the influence of his uncle's brother, Samuel Lysons, a junior clerkship in the branch Record Office at the Tower of London; it was, however, from Henry Petrie (who soon after this succeeded Lysons at the Tower) that he received his education as an archivist. On Petrie's retirement, the compilation of the "Monumenta Historica," published in 1848, was entrusted to him, and to this work he wrote the "General Introduction."