After being released on bail Mr Bush said he had done nothing wrong and that he was the subject of a "witch hunt".
Mr Bush is suspected of misusing of a government credit card and importing explosive devices without a permit.
The Cayman Islands is a British overseas territory. It is one of the world's largest financial centres and a well-known tax haven.
Mr Bush, 57, has been in power since his United Democratic Party (UDP) won general elections in 2009.
He also holds the posts of minister of finance, tourism and development.
"I have done nothing wrong and I shall not be resigning as premier," Mr Bush said in a statement.
"I also wish to assure one and all that the government continues to operate as normal."
Mr Bush was arrested on Tuesday and released on Wednesday after questioning.
On a trip to Jamaica on Thursday he suggested that he was being targeted by senior island officials appointed by the UK.
"It is nothing but a political, very vindictive political witch hunt," he was quoted as saying by the Associated Press news agency.
"I have made a lot of friends and I have made a lot of enemies.
"There are a lot of jealous people in a very small island."
Cayman Islands Premier McKeeva Bush says he will not resign despite his arrest on suspicion of corruption.