Any remote possibility of returning to the Whigs was scuttled by the Lichfield House Compact by which the Irish Repealers, Whigs and Radicals agreed to vote out Peel's government.
Any chance of rejoining the Whigs was dashed by the Lichfield House Agreement, which saw the Irish Repealers, Whigs, and Radicals agree to vote out Peel's administration.