Former chairman Stephen Green said HSBC should have done more due diligence before the purchase.
He said: "It would have been better to have drilled into the detail much earlier."
Lord Green was speaking to the House of Lords Economic Affairs Committee.
HSBC agreed to pay the authorities in Geneva 40 million Swiss francs (Â£28m) to settle the investigation into allegations of money laundering at the Swiss bank.
The payment was to compensate the authorities for past organisational failings and no criminal charges were filed.
Lord Green was chief executive and then chairman of HSBC between 2003 and 2010.
When asked by Lord Hollick how the bank had allowed "industrial scale" tax evasion at the Swiss private bank, Lord Green said: "I do not believe that anybody was aware of the - to use your phrase - industrial scale systems of tax evasion.
"We became aware of this situation, the extent of it, only in around 2010.
Lord Green also said that the bank failed to carry out due diligence on an acquisition in 2002 of Grupo Financiero Bital in Mexico.
He said: "Yes, with hindsight we should have focussed earlier on the anti-money laundering issues that were plainly in the bank. We should have done that. We didn't get it right."
Investigations by US authorities led to the UK-based bank being fined almost $2bn for failing to stop criminals using its banking systems to launder money.
Lord Green also admitted that the issue of high pay packages for bank employees had disturbed him: "It certainly kept me awake.
"For me this issue of remuneration was the most difficult one we wrestled with. How could anyone be comfortable with a situation where you've got very senior people being paid very large amounts of money or indeed quite young people being paid large amounts of money - an enormous multiple of, let's say the head of large inner city school?
"There's no possible way on moral grounds of justifying this. Did that leave me uncomfortable. You bet it did."
Former HSBC boss Lord Green has admitted the bank "didn't get everything right" when buying the Swiss private bank at the centre of a tax evasion scandal.