Hansom was born at 63 Micklegate, York (now #114, the Brigantes pub) to a large Roman Catholic family and baptised as Josephus Aloysius Handsom(e). He was the brother of the architect Charles Francis Hansom and the uncle of Edward J. Hansom. He was apprenticed to his father, Richard, as a joiner, but showing an early aptitude for draughtsmanship and construction, he was permitted to transfer his apprenticeship to a York architect named Matthew Philips. By 1829 he had completed his apprenticeship and became a clerk in Philips' office.