Halliday left the office of lieutenant-governor in May 1859. He had generally enjoyed the support of the governor-general during his tenure but he did antagonise some people within the official circles of the Indian government. Most notably, he had a long-running dispute with William Tayler, whom he considered to be incompetent. Halliday eventually removed Tayler from his position as Commissioner of Patna, following what Halliday considered to be a poor decision made by Tayler. This ultimately caused Tayler to resign and to bear a resentment that persisted until he died in 1892 and caused him publicly to seek redress for Halliday's actions.