In 1780 Gandon declined an invitation from a member of the Romanov family to work in St. Petersburg but in 1781, at the age of 38, he accepted an invitation to Ireland from Lord Carlow and John Beresford (the Revenue Commissioner for Ireland) to supervise the construction of the new Custom House in Dublin. Thomas Cooley, the original architect on that project, had died and Gandon was chosen to assume complete control. The new Custom House was unpopular with Dublin Corporation and some city merchants who complained that it moved the axis of the city, would leave little room for shipping, and it was being built on what at the time was a swamp. It is said that the Irish people were so opposed to the Custom House and its associated taxes that Beresford had to smuggle Gandon into the country and keep him hidden in his own home for the first three months. The project was eventually completed at a cost of £200,000, an enormous sum at the time.